[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:41 dist-test-slave-h7jz krb5kdc[19041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:41 dist-test-slave-h7jz krb5kdc[19041](info): set up 2 sockets
Jun 18 22:32:41 dist-test-slave-h7jz krb5kdc[19041](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260618 22:32:41.946741 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.000s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:32:42 dist-test-slave-h7jz krb5kdc[19041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:32:42.106124 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:46559
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:46559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:42.412583 19054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:42.412874 19054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:42.412958 19054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:42.420090 19054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:42.420203 19054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:42.420254 19054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:42.420307 19054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:42.420349 19054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:42.430161 19054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:46559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:46559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:42.431414 19054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:42.432686 19054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:42.444106 19059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:42.444164 19060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:42.444131 19062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:42.444906 19054 server_base.cc:1034] running on GCE node
I20260618 22:32:42.445641 19054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:42.447080 19054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:42.448280 19054 hybrid_clock.cc:648] HybridClock initialized: now 1781821962448218 us; error 51 us; skew 500 ppm
I20260618 22:32:42.448609 19054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:42 dist-test-slave-h7jz krb5kdc[19041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821962, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:42.451597 19054 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:32:42.453061 19054 webserver.cc:460] Webserver started at http://127.18.150.190:42109/ using document root <none> and password file <none>
I20260618 22:32:42.453701 19054 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:42.453796 19054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:42.454139 19054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:42.456945 19054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "1b7fb45f7d0e4027b02b01982de886d9"
format_stamp: "Formatted at 2026-06-18 22:32:42 on dist-test-slave-h7jz"
server_key: "338fcca1cae08914a2844d0d2c5818fc"
server_key_iv: "230e1597503f27a7afa39d19be1182f6"
server_key_version: "encryptionkey@0"
I20260618 22:32:42.457646 19054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "1b7fb45f7d0e4027b02b01982de886d9"
format_stamp: "Formatted at 2026-06-18 22:32:42 on dist-test-slave-h7jz"
server_key: "338fcca1cae08914a2844d0d2c5818fc"
server_key_iv: "230e1597503f27a7afa39d19be1182f6"
server_key_version: "encryptionkey@0"
I20260618 22:32:42.462544 19054 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260618 22:32:42.465776 19069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:42.467108 19054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:32:42.467262 19054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "1b7fb45f7d0e4027b02b01982de886d9"
format_stamp: "Formatted at 2026-06-18 22:32:42 on dist-test-slave-h7jz"
server_key: "338fcca1cae08914a2844d0d2c5818fc"
server_key_iv: "230e1597503f27a7afa39d19be1182f6"
server_key_version: "encryptionkey@0"
I20260618 22:32:42.467422 19054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:42.504741 19054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:42.508596 19054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:42.508843 19054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:42.525098 19054 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:46559
I20260618 22:32:42.525105 19120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:46559 every 8 connection(s)
I20260618 22:32:42.526469 19054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:42.530093 19121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:42.531052 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 19054
I20260618 22:32:42.531327 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:32:42.533113 19034 external_mini_cluster.cc:1428] Setting key 19a5e68be0caa33e88ae6727067232d6
I20260618 22:32:42.540843 19121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9: Bootstrap starting.
I20260618 22:32:42.543754 19121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:42.544752 19121 log.cc:824] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:42.547570 19121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9: No bootstrap required, opened a new log
I20260618 22:32:42.554862 19121 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7fb45f7d0e4027b02b01982de886d9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46559 } }
I20260618 22:32:42.555259 19121 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:42.555364 19121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7fb45f7d0e4027b02b01982de886d9, State: Initialized, Role: FOLLOWER
I20260618 22:32:42.555927 19121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7fb45f7d0e4027b02b01982de886d9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46559 } }
I20260618 22:32:42.556136 19121 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:42.556255 19121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:42.556406 19121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:42.558557 19121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7fb45f7d0e4027b02b01982de886d9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46559 } }
I20260618 22:32:42.559031 19121 leader_election.cc:302] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b7fb45f7d0e4027b02b01982de886d9; no voters:
Jun 18 22:32:42 dist-test-slave-h7jz krb5kdc[19041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:42.559664 19121 leader_election.cc:288] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:42.559762 19126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:42.561480 19126 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [term 1 LEADER]: Becoming Leader. State: Replica: 1b7fb45f7d0e4027b02b01982de886d9, State: Running, Role: LEADER
I20260618 22:32:42.562054 19121 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:42.562099 19126 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7fb45f7d0e4027b02b01982de886d9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46559 } }
I20260618 22:32:42.569283 19123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:42.536860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56196 (local address 127.18.150.190:46559)
0618 22:32:42.537602 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:42.537614 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:42.538835 (+ 1221us) server_negotiation.cc:407] Connection header received
0618 22:32:42.547373 (+ 8538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:42.547396 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:42.547793 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:42.548164 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:42.550970 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:42.552484 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:42.554204 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:42.554844 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:42.555004 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:42.560668 (+ 5664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:42.560679 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:42.560700 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:42.560752 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:42.564363 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:42.565228 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:42.565235 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:42.565241 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:42.565396 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:42.565839 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:42.565844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:42.565848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:42.566392 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:42.566637 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:42.567640 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0618 22:32:42.568019 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":338,"threads_started":1}
I20260618 22:32:42.573699 19127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b7fb45f7d0e4027b02b01982de886d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7fb45f7d0e4027b02b01982de886d9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46559 } } }
I20260618 22:32:42.574103 19127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:42.574198 19128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b7fb45f7d0e4027b02b01982de886d9. Latest consensus state: current_term: 1 leader_uuid: "1b7fb45f7d0e4027b02b01982de886d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7fb45f7d0e4027b02b01982de886d9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46559 } } }
I20260618 22:32:42.574478 19128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:42.578799 19136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:42.583230 19136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:42.591060 19136 catalog_manager.cc:1347] Generated new cluster ID: 0565159a20e8425ab9ef1b98c216868d
I20260618 22:32:42.591171 19136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:42.607983 19136 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:42.609860 19136 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:42.630147 19136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b7fb45f7d0e4027b02b01982de886d9: Generated new TSK 0
I20260618 22:32:42.631219 19136 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:32:42.681730 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46559
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:42.992723 19148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:42.992998 19148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:42.993093 19148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:43.000221 19148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:43.000329 19148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:43.000454 19148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:32:43.010007 19148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:43.011206 19148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:43.012480 19148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:43.024986 19156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:43.025079 19153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:43.025106 19154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:43.025628 19148 server_base.cc:1034] running on GCE node
I20260618 22:32:43.026261 19148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:43.027171 19148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:43.028380 19148 hybrid_clock.cc:648] HybridClock initialized: now 1781821963028331 us; error 44 us; skew 500 ppm
I20260618 22:32:43.028684 19148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:43 dist-test-slave-h7jz krb5kdc[19041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:43.031683 19148 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:32:43.033042 19148 webserver.cc:460] Webserver started at http://127.18.150.129:44159/ using document root <none> and password file <none>
I20260618 22:32:43.033667 19148 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:43.033757 19148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:43.034056 19148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:43.036823 19148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "4ec802bf381544679a94716a4d6bea96"
format_stamp: "Formatted at 2026-06-18 22:32:43 on dist-test-slave-h7jz"
server_key: "92adb9231362bdb6b3f2b8902fbc2611"
server_key_iv: "e65abc2143127c74380886b2292b7b98"
server_key_version: "encryptionkey@0"
I20260618 22:32:43.037505 19148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "4ec802bf381544679a94716a4d6bea96"
format_stamp: "Formatted at 2026-06-18 22:32:43 on dist-test-slave-h7jz"
server_key: "92adb9231362bdb6b3f2b8902fbc2611"
server_key_iv: "e65abc2143127c74380886b2292b7b98"
server_key_version: "encryptionkey@0"
I20260618 22:32:43.042364 19148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260618 22:32:43.045704 19163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:43.047091 19148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:32:43.047227 19148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "4ec802bf381544679a94716a4d6bea96"
format_stamp: "Formatted at 2026-06-18 22:32:43 on dist-test-slave-h7jz"
server_key: "92adb9231362bdb6b3f2b8902fbc2611"
server_key_iv: "e65abc2143127c74380886b2292b7b98"
server_key_version: "encryptionkey@0"
I20260618 22:32:43.047402 19148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:43.065833 19148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:43.070139 19148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:43.070657 19148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:43.071640 19148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:32:43.072953 19148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:43.073027 19148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:43.073148 19148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:43.073199 19148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:43.103009 19148 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:40721
I20260618 22:32:43.103055 19275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:40721 every 8 connection(s)
I20260618 22:32:43.104296 19148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:32:43.106482 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 19148
I20260618 22:32:43.106668 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:32:43.107375 19034 external_mini_cluster.cc:1428] Setting key b88793093948979c99d892ba05960c3b
Jun 18 22:32:43 dist-test-slave-h7jz krb5kdc[19041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
I20260618 22:32:43.123952 19278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:43.106840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:40981 (local address 127.18.150.190:46559)
0618 22:32:43.107293 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:43.107299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:43.108296 (+ 997us) server_negotiation.cc:407] Connection header received
0618 22:32:43.110301 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:43.110306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:43.110377 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:43.110527 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:43.111955 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:43.112940 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:43.114534 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:43.114931 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:43.115044 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:43.118374 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:43.118379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:43.118382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:43.118423 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:43.120857 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:43.121603 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:43.121607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:43.121609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:43.121722 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:43.122226 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:43.122230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:43.122232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:43.122503 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:43.122630 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:43.123369 (+ 739us) server_negotiation.cc:299] Negotiation successful
0618 22:32:43.123590 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":220,"threads_started":1}
I20260618 22:32:43.124972 19279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:43.107358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46559 (local address 127.18.150.129:40981)
0618 22:32:43.108142 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:43.108187 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:43.110046 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:43.110709 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:43.110717 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:43.111228 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:43.111792 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:43.111809 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:43.113091 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:43.113095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:43.114390 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:43.114403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:43.115157 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:43.115164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:43.115392 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:43.116027 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:43.116054 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:43.118151 (+ 2097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:43.121000 (+ 2849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:43.121010 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:43.121030 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:43.121473 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:43.121829 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:43.121834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:43.121838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:43.122111 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:43.122625 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:43.122632 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:43.123021 (+ 389us) client_negotiation.cc:769] Sending connection context
0618 22:32:43.123261 (+ 240us) client_negotiation.cc:240] Negotiation successful
0618 22:32:43.123795 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":310,"threads_started":1}
I20260618 22:32:43.126579 19276 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46559
I20260618 22:32:43.126904 19276 heartbeater.cc:464] Registering TS with master...
I20260618 22:32:43.127661 19276 heartbeater.cc:511] Master 127.18.150.190:46559 requested a full tablet report, sending...
I20260618 22:32:43.130036 19086 ts_manager.cc:194] Registered new tserver with Master: 4ec802bf381544679a94716a4d6bea96 (127.18.150.129:40721)
I20260618 22:32:43.131971 19086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:40981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:32:43.145040 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46559
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:43.451069 19282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:43.451352 19282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:43.451483 19282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:43.458684 19282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:43.458791 19282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:43.458904 19282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:32:43.468995 19282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:43.470260 19282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:43.471524 19282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:43.484179 19287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:43.484127 19290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:43.484221 19288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:43.485378 19282 server_base.cc:1034] running on GCE node
I20260618 22:32:43.485931 19282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:43.486874 19282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:43.488067 19282 hybrid_clock.cc:648] HybridClock initialized: now 1781821963488001 us; error 61 us; skew 500 ppm
I20260618 22:32:43.488350 19282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:43 dist-test-slave-h7jz krb5kdc[19041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:43.491308 19282 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:32:43.492673 19282 webserver.cc:460] Webserver started at http://127.18.150.130:36481/ using document root <none> and password file <none>
I20260618 22:32:43.493297 19282 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:43.493409 19282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:43.493688 19282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:43.496455 19282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "46032b9f00b7494ba8dd3133ae0be9c4"
format_stamp: "Formatted at 2026-06-18 22:32:43 on dist-test-slave-h7jz"
server_key: "b0afc547fdfc92be159eb3b1e371e5cd"
server_key_iv: "3d74c1a3bd78a0d1d2fb4a73be2332a0"
server_key_version: "encryptionkey@0"
I20260618 22:32:43.497143 19282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "46032b9f00b7494ba8dd3133ae0be9c4"
format_stamp: "Formatted at 2026-06-18 22:32:43 on dist-test-slave-h7jz"
server_key: "b0afc547fdfc92be159eb3b1e371e5cd"
server_key_iv: "3d74c1a3bd78a0d1d2fb4a73be2332a0"
server_key_version: "encryptionkey@0"
I20260618 22:32:43.502089 19282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260618 22:32:43.505446 19297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:43.506862 19282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:32:43.507033 19282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "46032b9f00b7494ba8dd3133ae0be9c4"
format_stamp: "Formatted at 2026-06-18 22:32:43 on dist-test-slave-h7jz"
server_key: "b0afc547fdfc92be159eb3b1e371e5cd"
server_key_iv: "3d74c1a3bd78a0d1d2fb4a73be2332a0"
server_key_version: "encryptionkey@0"
I20260618 22:32:43.507221 19282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:43.524367 19282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:43.529841 19282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:43.530187 19282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:43.531395 19282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:32:43.532998 19282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:43.533100 19282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:43.533218 19282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:43.533277 19282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:43.563819 19282 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:40395
I20260618 22:32:43.563908 19409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:40395 every 8 connection(s)
I20260618 22:32:43.565084 19282 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:32:43.570442 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 19282
I20260618 22:32:43.570640 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:32:43.571151 19034 external_mini_cluster.cc:1428] Setting key 9a85ef6dd7d6b8943fb4999bc95bcfe7
Jun 18 22:32:43 dist-test-slave-h7jz krb5kdc[19041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:43.584589 19278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:43.567508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:53043 (local address 127.18.150.190:46559)
0618 22:32:43.567679 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:43.567683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:43.568767 (+ 1084us) server_negotiation.cc:407] Connection header received
0618 22:32:43.570603 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:43.570609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:43.570695 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:43.570805 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:43.572446 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:43.573368 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:43.574937 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:43.575500 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:43.575622 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:43.579016 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:43.579021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:43.579024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:43.579055 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:43.581425 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:43.582307 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:43.582311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:43.582313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:43.582435 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:43.582883 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:43.582886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:43.582887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:43.583115 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:43.583220 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:43.584093 (+ 873us) server_negotiation.cc:299] Negotiation successful
0618 22:32:43.584302 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:32:43.585995 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:43.567918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46559 (local address 127.18.150.130:53043)
0618 22:32:43.568599 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:43.568641 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:43.570288 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:43.571037 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:43.571046 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:43.571589 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:43.572281 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:43.572307 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:43.573515 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:43.573519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:43.574794 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:43.574807 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:43.575691 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:43.575696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:43.575919 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:43.576640 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:43.576666 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:43.578809 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:43.581580 (+ 2771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:43.581589 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:43.581607 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:43.582198 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:43.582538 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:43.582542 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:43.582546 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:43.582783 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:43.583230 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:43.583238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:43.583684 (+ 446us) client_negotiation.cc:769] Sending connection context
0618 22:32:43.583989 (+ 305us) client_negotiation.cc:240] Negotiation successful
0618 22:32:43.584558 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":248,"threads_started":1}
I20260618 22:32:43.587409 19410 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46559
I20260618 22:32:43.587769 19410 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
I20260618 22:32:43.588559 19410 heartbeater.cc:511] Master 127.18.150.190:46559 requested a full tablet report, sending...
I20260618 22:32:43.590022 19086 ts_manager.cc:194] Registered new tserver with Master: 46032b9f00b7494ba8dd3133ae0be9c4 (127.18.150.130:40395)
I20260618 22:32:43.591005 19086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:53043
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:32:43.608911 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46559
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:43.927299 19415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:43.927582 19415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:43.927676 19415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:43.934767 19415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:43.934877 19415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:43.934990 19415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:32:43.944840 19415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:43.946193 19415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:43.947466 19415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:43.960227 19420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:43.960201 19421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:43.960173 19423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:43.960459 19415 server_base.cc:1034] running on GCE node
I20260618 22:32:43.961329 19415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:43.962327 19415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:43.963550 19415 hybrid_clock.cc:648] HybridClock initialized: now 1781821963963482 us; error 51 us; skew 500 ppm
I20260618 22:32:43.963899 19415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:43 dist-test-slave-h7jz krb5kdc[19041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:43.967072 19415 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:32:43.968464 19415 webserver.cc:460] Webserver started at http://127.18.150.131:41455/ using document root <none> and password file <none>
I20260618 22:32:43.969163 19415 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:43.969272 19415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:43.969558 19415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:43.972556 19415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "84762966589f4eebaa4e26fb1c981b9b"
format_stamp: "Formatted at 2026-06-18 22:32:43 on dist-test-slave-h7jz"
server_key: "51f154a765538903bd63544b563859dd"
server_key_iv: "8ffc0d3c0c3bba2f2a9029a24e5ff547"
server_key_version: "encryptionkey@0"
I20260618 22:32:43.973309 19415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "84762966589f4eebaa4e26fb1c981b9b"
format_stamp: "Formatted at 2026-06-18 22:32:43 on dist-test-slave-h7jz"
server_key: "51f154a765538903bd63544b563859dd"
server_key_iv: "8ffc0d3c0c3bba2f2a9029a24e5ff547"
server_key_version: "encryptionkey@0"
I20260618 22:32:43.978456 19415 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260618 22:32:43.982071 19430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:43.983613 19415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:32:43.983771 19415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "84762966589f4eebaa4e26fb1c981b9b"
format_stamp: "Formatted at 2026-06-18 22:32:43 on dist-test-slave-h7jz"
server_key: "51f154a765538903bd63544b563859dd"
server_key_iv: "8ffc0d3c0c3bba2f2a9029a24e5ff547"
server_key_version: "encryptionkey@0"
I20260618 22:32:43.983963 19415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:44.004537 19415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:44.008538 19415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:44.008797 19415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:44.009734 19415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:32:44.011142 19415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:44.011214 19415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:44.011355 19415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:44.011418 19415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:44.040889 19415 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:39069
I20260618 22:32:44.040933 19542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:39069 every 8 connection(s)
I20260618 22:32:44.042476 19415 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:32:44.044584 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 19415
I20260618 22:32:44.044773 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:32:44.045300 19034 external_mini_cluster.cc:1428] Setting key 7bdb7e8d4f79a32997497e617c1273f7
Jun 18 22:32:44 dist-test-slave-h7jz krb5kdc[19041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:44.061317 19278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:44.044990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:57903 (local address 127.18.150.190:46559)
0618 22:32:44.045154 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:44.045158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:44.046587 (+ 1429us) server_negotiation.cc:407] Connection header received
0618 22:32:44.048534 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:44.048539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:44.048618 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:44.048697 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:44.050141 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.050862 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.052350 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.052756 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.052862 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:44.056037 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:44.056042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:44.056045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:44.056074 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:44.058497 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.059155 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.059158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.059161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.059276 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.059693 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.059699 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.059702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.060014 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:44.060152 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:44.060786 (+ 634us) server_negotiation.cc:299] Negotiation successful
0618 22:32:44.060994 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:32:44.062304 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:44.045600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46559 (local address 127.18.150.131:57903)
0618 22:32:44.046427 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:44.046473 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:44.048256 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:44.048948 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:44.048956 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:44.049427 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:44.049981 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:44.049998 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.051003 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.051008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:44.052210 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:44.052220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.052974 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.052981 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:44.053204 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:44.053778 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:44.053802 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:44.055847 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:44.058620 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:44.058630 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:44.058647 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:44.059035 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:44.059375 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:44.059380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:44.059384 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:44.059600 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:44.060132 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:44.060139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:44.060452 (+ 313us) client_negotiation.cc:769] Sending connection context
0618 22:32:44.060675 (+ 223us) client_negotiation.cc:240] Negotiation successful
0618 22:32:44.061169 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":314,"threads_started":1}
I20260618 22:32:44.063624 19543 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46559
I20260618 22:32:44.063889 19543 heartbeater.cc:464] Registering TS with master...
I20260618 22:32:44.064517 19543 heartbeater.cc:511] Master 127.18.150.190:46559 requested a full tablet report, sending...
I20260618 22:32:44.065784 19086 ts_manager.cc:194] Registered new tserver with Master: 84762966589f4eebaa4e26fb1c981b9b (127.18.150.131:39069)
I20260618 22:32:44.066676 19086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:57903
I20260618 22:32:44.073302 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:32:44 dist-test-slave-h7jz krb5kdc[19041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821964, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:32:44 dist-test-slave-h7jz krb5kdc[19041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821964, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:32:44.114773 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:44.099637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46376 (local address 127.18.150.129:40721)
0618 22:32:44.100178 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:44.100185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:44.100251 (+ 66us) server_negotiation.cc:407] Connection header received
0618 22:32:44.100401 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:44.100407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:44.100634 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:44.100768 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:44.102068 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.103133 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.104906 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.105453 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.105545 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:44.109827 (+ 4282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:44.109837 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:44.109849 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:44.109876 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:44.112470 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.113020 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.113027 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.113029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.113136 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.113445 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.113449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.113451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.113768 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:44.114062 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:44.114243 (+ 181us) server_negotiation.cc:299] Negotiation successful
0618 22:32:44.114422 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":285,"threads_started":1}
W20260618 22:32:44.115850 19191 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46376
Jun 18 22:32:44 dist-test-slave-h7jz krb5kdc[19041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260618 22:32:44.135257 19276 heartbeater.cc:503] Master 127.18.150.190:46559 was elected leader, sending a full tablet report...
Jun 18 22:32:44 dist-test-slave-h7jz krb5kdc[19041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:32:44.155211 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:44.141538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46384 (local address 127.18.150.129:40721)
0618 22:32:44.141770 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:44.141777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:44.142098 (+ 321us) server_negotiation.cc:407] Connection header received
0618 22:32:44.142284 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:44.142288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:44.142411 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:44.142501 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:44.143822 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.144531 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.145771 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.146237 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.146367 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:44.150115 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:44.150120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:44.150123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:44.150159 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:44.152755 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.153361 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.153366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.153368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.153503 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.153857 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.153861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.153863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.154199 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:44.154403 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:44.154706 (+ 303us) server_negotiation.cc:299] Negotiation successful
0618 22:32:44.154908 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260618 22:32:44.156491 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 19148
I20260618 22:32:44.168836 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:44.157386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46394 (local address 127.18.150.129:40721)
0618 22:32:44.157540 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:44.157544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:44.157849 (+ 305us) server_negotiation.cc:407] Connection header received
0618 22:32:44.157991 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:44.157994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:44.158049 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:44.158121 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:44.159180 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.159846 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.161196 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.161584 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.161668 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:44.164262 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:44.164267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:44.164270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:44.164298 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:44.166686 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.167272 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.167276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.167278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.167386 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.167692 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.167695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.167697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.167929 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:44.168086 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:44.168348 (+ 262us) server_negotiation.cc:299] Negotiation successful
0618 22:32:44.168495 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:32:44.169564 19271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:44.251317 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 19148
I20260618 22:32:44.267688 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 19282
Jun 18 22:32:44 dist-test-slave-h7jz krb5kdc[19041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:32:44.282732 19557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:44.268485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.18.150.130:40395)
0618 22:32:44.269008 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:44.269014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:44.269040 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:32:44.269106 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:44.269113 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:44.269309 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:44.269440 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:44.270698 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.271733 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.272960 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.273536 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.273648 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:44.277362 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:44.277379 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:44.277399 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:44.277442 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:44.280202 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.280778 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.280787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.280791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.280904 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.281264 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.281270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.281273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.281594 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:44.281860 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:44.282181 (+ 321us) server_negotiation.cc:299] Negotiation successful
0618 22:32:44.282413 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":269,"threads_started":1}
I20260618 22:32:44.283949 19405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:44.363921 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 19282
I20260618 22:32:44.378173 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 19415
Jun 18 22:32:44 dist-test-slave-h7jz krb5kdc[19041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:32:44.394304 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:44.379231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52856 (local address 127.18.150.131:39069)
0618 22:32:44.379850 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:44.379857 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:44.379883 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:32:44.379945 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:44.379952 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:44.380177 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:44.380303 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:44.381487 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.382794 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.384099 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:44.384643 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:44.384736 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:44.388669 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:44.388679 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:44.388694 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:44.388733 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:44.391634 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.392207 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.392214 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.392216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.392322 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:44.392659 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:44.392665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:44.392668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:44.393134 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:44.393434 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:44.393733 (+ 299us) server_negotiation.cc:299] Negotiation successful
0618 22:32:44.393922 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":344,"threads_started":1}
I20260618 22:32:44.395496 19538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:44.475040 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 19415
I20260618 22:32:44.487481 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 19054
I20260618 22:32:44.488672 19116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:44.588567 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 19054
2026-06-18T22:32:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3661 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:44 dist-test-slave-h7jz krb5kdc[19564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:44 dist-test-slave-h7jz krb5kdc[19564](info): set up 2 sockets
Jun 18 22:32:44 dist-test-slave-h7jz krb5kdc[19564](info): commencing operation
krb5kdc: starting...
W20260618 22:32:45.573539 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:32:45 dist-test-slave-h7jz krb5kdc[19564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:32:45.742067 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:40407
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:40407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:46.055296 19577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:46.055581 19577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:46.055707 19577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:46.062680 19577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:46.062775 19577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:46.062845 19577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:46.062922 19577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:46.062978 19577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:46.072700 19577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:40407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:40407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:46.073989 19577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:46.075285 19577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:46.086405 19582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:46.086488 19583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:46.086522 19585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:46.087529 19577 server_base.cc:1034] running on GCE node
I20260618 22:32:46.088332 19577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:46.089774 19577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:46.090989 19577 hybrid_clock.cc:648] HybridClock initialized: now 1781821966090923 us; error 61 us; skew 500 ppm
I20260618 22:32:46.091328 19577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:46 dist-test-slave-h7jz krb5kdc[19564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821966, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:46.096899 19577 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:32:46.098465 19577 webserver.cc:460] Webserver started at http://127.18.150.190:34781/ using document root <none> and password file <none>
I20260618 22:32:46.099135 19577 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:46.099246 19577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:46.099551 19577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:46.102532 19577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "1ae38f18615a4789a6bf485106e11bc3"
format_stamp: "Formatted at 2026-06-18 22:32:46 on dist-test-slave-h7jz"
server_key: "b236088481864b02d7a3cfa92ef581ff"
server_key_iv: "3a944a5baa985473388d0faa26f6166c"
server_key_version: "encryptionkey@0"
I20260618 22:32:46.103229 19577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "1ae38f18615a4789a6bf485106e11bc3"
format_stamp: "Formatted at 2026-06-18 22:32:46 on dist-test-slave-h7jz"
server_key: "b236088481864b02d7a3cfa92ef581ff"
server_key_iv: "3a944a5baa985473388d0faa26f6166c"
server_key_version: "encryptionkey@0"
I20260618 22:32:46.108273 19577 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:32:46.111609 19592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:46.112985 19577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:32:46.113147 19577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "1ae38f18615a4789a6bf485106e11bc3"
format_stamp: "Formatted at 2026-06-18 22:32:46 on dist-test-slave-h7jz"
server_key: "b236088481864b02d7a3cfa92ef581ff"
server_key_iv: "3a944a5baa985473388d0faa26f6166c"
server_key_version: "encryptionkey@0"
I20260618 22:32:46.113319 19577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:46.130764 19577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:46.135084 19577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:46.135345 19577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:46.152360 19577 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:40407
I20260618 22:32:46.152359 19643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:40407 every 8 connection(s)
I20260618 22:32:46.153671 19577 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:46.157320 19644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:46.159446 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 19577
I20260618 22:32:46.159648 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:32:46.160249 19034 external_mini_cluster.cc:1428] Setting key 981c22aeabac6128fd89e58304dfabd5
I20260618 22:32:46.169514 19644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3: Bootstrap starting.
I20260618 22:32:46.172922 19644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:46.173863 19644 log.cc:824] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:32:46 dist-test-slave-h7jz krb5kdc[19564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:46.176728 19644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3: No bootstrap required, opened a new log
I20260618 22:32:46.182847 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:46.162115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.18.150.190:40407)
0618 22:32:46.162918 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:46.162933 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:46.162982 (+ 49us) server_negotiation.cc:407] Connection header received
0618 22:32:46.165165 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:46.165193 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:46.165697 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:46.166162 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:46.167350 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:46.169210 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:46.170505 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:46.171564 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:46.171734 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:46.175605 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:46.175613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:46.175628 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:46.175664 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:46.178772 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:46.179451 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:46.179458 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:46.179468 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:46.179627 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:46.179988 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:46.179991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:46.179993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:46.180526 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:46.180835 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:46.181336 (+ 501us) server_negotiation.cc:299] Negotiation successful
0618 22:32:46.181732 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":313,"threads_started":1}
I20260618 22:32:46.184664 19644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae38f18615a4789a6bf485106e11bc3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40407 } }
I20260618 22:32:46.185086 19644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:46.185195 19644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ae38f18615a4789a6bf485106e11bc3, State: Initialized, Role: FOLLOWER
I20260618 22:32:46.185799 19644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae38f18615a4789a6bf485106e11bc3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40407 } }
I20260618 22:32:46.186081 19644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:46.186203 19644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:46.186347 19644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:46.188551 19644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae38f18615a4789a6bf485106e11bc3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40407 } }
I20260618 22:32:46.189124 19644 leader_election.cc:302] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ae38f18615a4789a6bf485106e11bc3; no voters:
I20260618 22:32:46.189723 19644 leader_election.cc:288] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:46.189837 19649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:46.190516 19649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [term 1 LEADER]: Becoming Leader. State: Replica: 1ae38f18615a4789a6bf485106e11bc3, State: Running, Role: LEADER
I20260618 22:32:46.191002 19649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae38f18615a4789a6bf485106e11bc3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40407 } }
I20260618 22:32:46.191696 19644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:46.193581 19650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ae38f18615a4789a6bf485106e11bc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae38f18615a4789a6bf485106e11bc3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40407 } } }
I20260618 22:32:46.194372 19650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:46.193965 19651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ae38f18615a4789a6bf485106e11bc3. Latest consensus state: current_term: 1 leader_uuid: "1ae38f18615a4789a6bf485106e11bc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae38f18615a4789a6bf485106e11bc3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40407 } } }
I20260618 22:32:46.195271 19651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:46.199851 19655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:46.205142 19655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:46.213675 19655 catalog_manager.cc:1347] Generated new cluster ID: 14d21752d1d2451db8ec2b22882d7959
I20260618 22:32:46.213783 19655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:46.238106 19655 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:46.239383 19655 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:46.250641 19655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ae38f18615a4789a6bf485106e11bc3: Generated new TSK 0
I20260618 22:32:46.251472 19655 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:32:46.290800 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40407
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:46.600005 19670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:46.600289 19670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:46.600382 19670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:46.607899 19670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:46.608049 19670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:46.608170 19670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:32:46.618714 19670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40407
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:46.620000 19670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:46.621289 19670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:46.633805 19678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:46.633872 19675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:46.633833 19676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:46.634344 19670 server_base.cc:1034] running on GCE node
I20260618 22:32:46.635026 19670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:46.635890 19670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:46.637081 19670 hybrid_clock.cc:648] HybridClock initialized: now 1781821966637030 us; error 45 us; skew 500 ppm
I20260618 22:32:46.637362 19670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:46 dist-test-slave-h7jz krb5kdc[19564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821966, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:46.640314 19670 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:32:46.641661 19670 webserver.cc:460] Webserver started at http://127.18.150.129:37833/ using document root <none> and password file <none>
I20260618 22:32:46.642326 19670 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:46.642443 19670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:46.642716 19670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:46.645466 19670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "aebc573bc81347039dfc827a0b15f039"
format_stamp: "Formatted at 2026-06-18 22:32:46 on dist-test-slave-h7jz"
server_key: "4e3ee40e67ac71ad9573fd659246803f"
server_key_iv: "f686f546287d82eaa94aa3559bd38c15"
server_key_version: "encryptionkey@0"
I20260618 22:32:46.646210 19670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "aebc573bc81347039dfc827a0b15f039"
format_stamp: "Formatted at 2026-06-18 22:32:46 on dist-test-slave-h7jz"
server_key: "4e3ee40e67ac71ad9573fd659246803f"
server_key_iv: "f686f546287d82eaa94aa3559bd38c15"
server_key_version: "encryptionkey@0"
I20260618 22:32:46.650906 19670 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260618 22:32:46.654026 19685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:46.655288 19670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:32:46.655441 19670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "aebc573bc81347039dfc827a0b15f039"
format_stamp: "Formatted at 2026-06-18 22:32:46 on dist-test-slave-h7jz"
server_key: "4e3ee40e67ac71ad9573fd659246803f"
server_key_iv: "f686f546287d82eaa94aa3559bd38c15"
server_key_version: "encryptionkey@0"
I20260618 22:32:46.655603 19670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:46.689064 19670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:46.693094 19670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:46.693341 19670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:46.694283 19670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:32:46.695614 19670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:46.695696 19670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:46.695819 19670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:46.695869 19670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:46.724032 19670 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:43169
I20260618 22:32:46.724123 19797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:43169 every 8 connection(s)
I20260618 22:32:46.725279 19670 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:32:46.725930 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 19670
I20260618 22:32:46.726200 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:32:46.726853 19034 external_mini_cluster.cc:1428] Setting key 6414ce244d865b87bf59d74fb86caa15
Jun 18 22:32:46 dist-test-slave-h7jz krb5kdc[19564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821966, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
I20260618 22:32:46.745738 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:46.727746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:42057 (local address 127.18.150.190:40407)
0618 22:32:46.728170 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:46.728176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:46.729391 (+ 1215us) server_negotiation.cc:407] Connection header received
0618 22:32:46.731955 (+ 2564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:46.731961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:46.732068 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:46.732183 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:46.733800 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:46.734556 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:46.736037 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:46.736423 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:46.736529 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:46.740063 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:46.740068 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:46.740071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:46.740107 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:46.742617 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:46.743344 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:46.743350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:46.743354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:46.743493 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:46.743930 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:46.743936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:46.743940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:46.744248 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:46.744388 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:46.745102 (+ 714us) server_negotiation.cc:299] Negotiation successful
0618 22:32:46.745328 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":215,"threads_started":1}
I20260618 22:32:46.746837 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:46.728463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:40407 (local address 127.18.150.129:42057)
0618 22:32:46.729223 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:46.729275 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:46.731613 (+ 2338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:46.732576 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:46.732585 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:46.733068 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:46.733640 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:46.733657 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:46.734700 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:46.734704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:46.735906 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:46.735918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:46.736636 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:46.736643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:46.736877 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:46.737531 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:46.737560 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:46.739820 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:46.742762 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:46.742772 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:46.742790 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:46.743219 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:46.743603 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:46.743609 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:46.743613 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:46.743819 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:46.744383 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:46.744392 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:46.744744 (+ 352us) client_negotiation.cc:769] Sending connection context
0618 22:32:46.745018 (+ 274us) client_negotiation.cc:240] Negotiation successful
0618 22:32:46.745570 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":277,"threads_started":1}
I20260618 22:32:46.748482 19798 heartbeater.cc:347] Connected to a master server at 127.18.150.190:40407
I20260618 22:32:46.748778 19798 heartbeater.cc:464] Registering TS with master...
I20260618 22:32:46.749538 19798 heartbeater.cc:511] Master 127.18.150.190:40407 requested a full tablet report, sending...
I20260618 22:32:46.751742 19609 ts_manager.cc:194] Registered new tserver with Master: aebc573bc81347039dfc827a0b15f039 (127.18.150.129:43169)
I20260618 22:32:46.753234 19609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:42057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:32:46.765749 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40407
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:47.084636 19804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:47.084896 19804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:47.084995 19804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:47.092149 19804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:47.092253 19804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:47.092402 19804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:32:47.102254 19804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40407
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:47.103598 19804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:47.104919 19804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:47.118738 19809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:47.118721 19812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:47.118691 19810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:47.119582 19804 server_base.cc:1034] running on GCE node
I20260618 22:32:47.120222 19804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:47.121212 19804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:47.122412 19804 hybrid_clock.cc:648] HybridClock initialized: now 1781821967122338 us; error 64 us; skew 500 ppm
I20260618 22:32:47.122745 19804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:47 dist-test-slave-h7jz krb5kdc[19564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821967, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:47.125741 19804 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:32:47.127185 19804 webserver.cc:460] Webserver started at http://127.18.150.130:44879/ using document root <none> and password file <none>
I20260618 22:32:47.127854 19804 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:47.127964 19804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:47.128247 19804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:47.131075 19804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "be8ecbea3a594237b542f9c41d5bf2ad"
format_stamp: "Formatted at 2026-06-18 22:32:47 on dist-test-slave-h7jz"
server_key: "82992086e17c7ac121d9c028a2bae0e2"
server_key_iv: "5d3c7f2ded1d381a845aeeee9eecfaee"
server_key_version: "encryptionkey@0"
I20260618 22:32:47.131825 19804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "be8ecbea3a594237b542f9c41d5bf2ad"
format_stamp: "Formatted at 2026-06-18 22:32:47 on dist-test-slave-h7jz"
server_key: "82992086e17c7ac121d9c028a2bae0e2"
server_key_iv: "5d3c7f2ded1d381a845aeeee9eecfaee"
server_key_version: "encryptionkey@0"
I20260618 22:32:47.136649 19804 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260618 22:32:47.140085 19819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:47.141434 19804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260618 22:32:47.141597 19804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "be8ecbea3a594237b542f9c41d5bf2ad"
format_stamp: "Formatted at 2026-06-18 22:32:47 on dist-test-slave-h7jz"
server_key: "82992086e17c7ac121d9c028a2bae0e2"
server_key_iv: "5d3c7f2ded1d381a845aeeee9eecfaee"
server_key_version: "encryptionkey@0"
I20260618 22:32:47.141760 19804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:47.159021 19804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:47.163690 19804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:47.163950 19804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:47.165072 19804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:32:47.166633 19804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:47.166725 19804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:47.166846 19804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:47.166906 19804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:47.196040 19804 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:41185
I20260618 22:32:47.196079 19931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:41185 every 8 connection(s)
I20260618 22:32:47.197233 19804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:32:47.201062 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 19804
I20260618 22:32:47.201248 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:32:47.201802 19034 external_mini_cluster.cc:1428] Setting key a8b30aaccb5650eb0bf3ea028890cac8
Jun 18 22:32:47 dist-test-slave-h7jz krb5kdc[19564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821967, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:32:47.216406 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.199712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:38357 (local address 127.18.150.190:40407)
0618 22:32:47.199884 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.199889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.201075 (+ 1186us) server_negotiation.cc:407] Connection header received
0618 22:32:47.203333 (+ 2258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.203338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.203418 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.203544 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:47.205029 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.206059 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.207514 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.207914 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.208013 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.211180 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:47.211185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:47.211188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:47.211217 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:47.213614 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.214325 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.214330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.214333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.214453 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.214841 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.214845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.214847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.215093 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:47.215233 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.215876 (+ 643us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.216088 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:32:47.217449 19934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.200185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:40407 (local address 127.18.150.130:38357)
0618 22:32:47.200890 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:47.200949 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:47.203036 (+ 2087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:47.203793 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:47.203805 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:47.204317 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:47.204855 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.204871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.206187 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.206192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.207384 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.207395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.208120 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.208127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.208396 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.209008 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:47.209030 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:47.210979 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:47.213733 (+ 2754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:47.213742 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:47.213758 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:47.214222 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:47.214552 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:47.214556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:47.214560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:47.214733 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:47.215203 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:47.215212 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:47.215530 (+ 318us) client_negotiation.cc:769] Sending connection context
0618 22:32:47.215771 (+ 241us) client_negotiation.cc:240] Negotiation successful
0618 22:32:47.216304 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":249,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
I20260618 22:32:47.218957 19932 heartbeater.cc:347] Connected to a master server at 127.18.150.190:40407
I20260618 22:32:47.219307 19932 heartbeater.cc:464] Registering TS with master...
I20260618 22:32:47.220170 19932 heartbeater.cc:511] Master 127.18.150.190:40407 requested a full tablet report, sending...
I20260618 22:32:47.221756 19609 ts_manager.cc:194] Registered new tserver with Master: be8ecbea3a594237b542f9c41d5bf2ad (127.18.150.130:41185)
I20260618 22:32:47.222751 19609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:38357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:32:47.239151 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40407
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:47.546331 19937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:47.546607 19937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:47.546700 19937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:47.553817 19937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:47.553977 19937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:47.554201 19937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:32:47.564304 19937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40407
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:47.565514 19937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:47.566812 19937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:47.579064 19945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:47.579128 19943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:47.579101 19942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:47.579810 19937 server_base.cc:1034] running on GCE node
I20260618 22:32:47.580375 19937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:47.581270 19937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:47.582460 19937 hybrid_clock.cc:648] HybridClock initialized: now 1781821967582412 us; error 47 us; skew 500 ppm
I20260618 22:32:47.582769 19937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:47 dist-test-slave-h7jz krb5kdc[19564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821967, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:47.585700 19937 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:32:47.587167 19937 webserver.cc:460] Webserver started at http://127.18.150.131:42299/ using document root <none> and password file <none>
I20260618 22:32:47.587864 19937 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:47.587983 19937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:47.588266 19937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:47.591107 19937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "946dc4d9fc08442d85a98a7429f95240"
format_stamp: "Formatted at 2026-06-18 22:32:47 on dist-test-slave-h7jz"
server_key: "45e1e8b7d253ca64716f222cdbce23aa"
server_key_iv: "9bdeb3d0415d5b8e91e4eb0a6e1d30aa"
server_key_version: "encryptionkey@0"
I20260618 22:32:47.591840 19937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "946dc4d9fc08442d85a98a7429f95240"
format_stamp: "Formatted at 2026-06-18 22:32:47 on dist-test-slave-h7jz"
server_key: "45e1e8b7d253ca64716f222cdbce23aa"
server_key_iv: "9bdeb3d0415d5b8e91e4eb0a6e1d30aa"
server_key_version: "encryptionkey@0"
I20260618 22:32:47.596839 19937 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:32:47.600275 19952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:47.601612 19937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:32:47.601750 19937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "946dc4d9fc08442d85a98a7429f95240"
format_stamp: "Formatted at 2026-06-18 22:32:47 on dist-test-slave-h7jz"
server_key: "45e1e8b7d253ca64716f222cdbce23aa"
server_key_iv: "9bdeb3d0415d5b8e91e4eb0a6e1d30aa"
server_key_version: "encryptionkey@0"
I20260618 22:32:47.601925 19937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:47.628010 19937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:47.632440 19937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:47.632695 19937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:47.633654 19937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:32:47.635042 19937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:47.635125 19937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:47.635239 19937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:47.635289 19937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:47.663694 19937 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:39005
I20260618 22:32:47.663738 20064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:39005 every 8 connection(s)
I20260618 22:32:47.664922 19937 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:32:47.674808 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 19937
I20260618 22:32:47.674965 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:32:47.675352 19034 external_mini_cluster.cc:1428] Setting key 6fcbc29df879e04e5b450806f1e40980
Jun 18 22:32:47 dist-test-slave-h7jz krb5kdc[19564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821967, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:47.683470 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.667447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:53109 (local address 127.18.150.190:40407)
0618 22:32:47.667612 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.667616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.668771 (+ 1155us) server_negotiation.cc:407] Connection header received
0618 22:32:47.670597 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.670602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.670684 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.670769 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:47.672255 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.672979 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.674527 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.674932 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.675053 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.678792 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:47.678797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:47.678800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:47.678832 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:47.680998 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.681618 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.681621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.681624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.681733 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.682131 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.682134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.682136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.682399 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:47.682512 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.683068 (+ 556us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.683234 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:32:47.684813 20067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.667942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:40407 (local address 127.18.150.131:53109)
0618 22:32:47.668622 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:47.668664 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:47.670329 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:47.671007 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:47.671016 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:47.671527 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:47.672091 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.672109 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.673121 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.673125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.674378 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.674394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.675145 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.675152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.675364 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.676026 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:47.676055 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:47.678607 (+ 2552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:47.681116 (+ 2509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:47.681122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:47.681134 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:47.681515 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:47.681806 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:47.681810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:47.681812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:47.682037 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:47.682484 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:47.682489 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:47.682790 (+ 301us) client_negotiation.cc:769] Sending connection context
0618 22:32:47.682996 (+ 206us) client_negotiation.cc:240] Negotiation successful
0618 22:32:47.683372 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":259,"threads_started":1}
I20260618 22:32:47.686640 20065 heartbeater.cc:347] Connected to a master server at 127.18.150.190:40407
I20260618 22:32:47.686939 20065 heartbeater.cc:464] Registering TS with master...
I20260618 22:32:47.687618 20065 heartbeater.cc:511] Master 127.18.150.190:40407 requested a full tablet report, sending...
I20260618 22:32:47.689045 19609 ts_manager.cc:194] Registered new tserver with Master: 946dc4d9fc08442d85a98a7429f95240 (127.18.150.131:39005)
I20260618 22:32:47.690131 19609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:53109
I20260618 22:32:47.690604 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:32:47.708572 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.696788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.18.150.190:40407)
0618 22:32:47.697027 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.697033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.697373 (+ 340us) server_negotiation.cc:407] Connection header received
0618 22:32:47.697520 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.697524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.697583 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.697664 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:47.699071 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.699811 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.701149 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.701555 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.701647 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.703880 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:47.703884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:47.703887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:47.703916 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:47.706349 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.706930 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.706934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.706937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.707057 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.707314 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.707317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.707318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.707639 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:47.707805 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.708178 (+ 373us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.708331 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:32:47.715046 19609 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:32:47.722311 19609 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:32:47.750468 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.738318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:41185 (local address 127.0.0.1:58090)
0618 22:32:47.742744 (+ 4426us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:47.742758 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:47.742867 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:47.743526 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:47.743533 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:47.743589 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:47.743993 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.744005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.746155 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.746160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.748160 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.748171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.749446 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.749452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.749638 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.749691 (+ 53us) client_negotiation.cc:769] Sending connection context
0618 22:32:47.749810 (+ 119us) client_negotiation.cc:240] Negotiation successful
0618 22:32:47.750014 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4314,"mutex_wait_us":80,"spinlock_wait_cycles":8320,"thread_start_us":3794,"threads_started":1}
I20260618 22:32:47.751454 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.738267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58090 (local address 127.18.150.130:41185)
0618 22:32:47.742394 (+ 4127us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.742400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.742852 (+ 452us) server_negotiation.cc:407] Connection header received
0618 22:32:47.743019 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.743026 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.743286 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.744202 (+ 916us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:47.744457 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.746017 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.748295 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.749246 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.749397 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.749586 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.750982 (+ 1396us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.751155 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3995,"thread_start_us":3871,"threads_started":1}
I20260618 22:32:47.751601 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.737771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:39005 (local address 127.0.0.1:50146)
0618 22:32:47.742468 (+ 4697us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:47.742517 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:47.742715 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:47.744607 (+ 1892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:47.744612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:47.744634 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:47.744960 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.744971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.746744 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.746749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.749071 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.749085 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.750902 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.750909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.751022 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.751043 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:32:47.751126 (+ 83us) client_negotiation.cc:240] Negotiation successful
0618 22:32:47.751246 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4455,"mutex_wait_us":91,"thread_start_us":5293,"threads_started":1}
I20260618 22:32:47.751631 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.737932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.18.150.131:39005)
0618 22:32:47.738646 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.738656 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.742654 (+ 3998us) server_negotiation.cc:407] Connection header received
0618 22:32:47.743864 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.743873 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.744172 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.744324 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:47.745144 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.746587 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.749220 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.750534 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.750710 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.750869 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.751183 (+ 314us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.751335 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":342,"threads_started":1}
I20260618 22:32:47.752748 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.742926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43169 (local address 127.0.0.1:47292)
0618 22:32:47.743475 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:47.743490 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:47.743598 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:47.745162 (+ 1564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:47.745166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:47.745190 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:47.745474 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.745485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.747235 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.747240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.749351 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.749364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.751968 (+ 2604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.751974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.752147 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.752178 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:32:47.752265 (+ 87us) client_negotiation.cc:240] Negotiation successful
0618 22:32:47.752382 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":1,"spinlock_wait_cycles":25088,"thread_start_us":1189,"threads_started":1}
I20260618 22:32:47.752755 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.743127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47292 (local address 127.18.150.129:43169)
0618 22:32:47.744676 (+ 1549us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.744681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.744701 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:32:47.744767 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.744772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.744949 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.745112 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:47.745631 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.747124 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.749519 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.751284 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.751466 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.752005 (+ 539us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.752315 (+ 310us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.752468 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1420,"thread_start_us":1284,"threads_started":1}
I20260618 22:32:47.755167 20000 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac51ef3b7a64edd829bbda5b6bc0e3d (DEFAULT_TABLE table=test-table [id=e867c17986314d5e9294216b1e0c6e72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:47.755182 19867 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac51ef3b7a64edd829bbda5b6bc0e3d (DEFAULT_TABLE table=test-table [id=e867c17986314d5e9294216b1e0c6e72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:47.756032 19798 heartbeater.cc:503] Master 127.18.150.190:40407 was elected leader, sending a full tablet report...
I20260618 22:32:47.756465 20000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ac51ef3b7a64edd829bbda5b6bc0e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:47.756497 19867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ac51ef3b7a64edd829bbda5b6bc0e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:47.756259 19733 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac51ef3b7a64edd829bbda5b6bc0e3d (DEFAULT_TABLE table=test-table [id=e867c17986314d5e9294216b1e0c6e72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:47.757421 19733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ac51ef3b7a64edd829bbda5b6bc0e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:47.767220 20083 tablet_bootstrap.cc:492] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240: Bootstrap starting.
I20260618 22:32:47.770327 20083 tablet_bootstrap.cc:654] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:47.770308 20084 tablet_bootstrap.cc:492] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad: Bootstrap starting.
I20260618 22:32:47.771636 20083 log.cc:824] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:47.772104 20085 tablet_bootstrap.cc:492] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039: Bootstrap starting.
I20260618 22:32:47.774385 20084 tablet_bootstrap.cc:654] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:47.774946 20085 tablet_bootstrap.cc:654] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:47.775508 20084 log.cc:824] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:47.775620 20083 tablet_bootstrap.cc:492] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240: No bootstrap required, opened a new log
I20260618 22:32:47.775833 20085 log.cc:824] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:47.775861 20083 ts_tablet_manager.cc:1397] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260618 22:32:47.778002 20085 tablet_bootstrap.cc:492] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039: No bootstrap required, opened a new log
I20260618 22:32:47.778211 20085 ts_tablet_manager.cc:1397] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.005s
I20260618 22:32:47.784173 20084 tablet_bootstrap.cc:492] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad: No bootstrap required, opened a new log
I20260618 22:32:47.784431 20084 ts_tablet_manager.cc:1397] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260618 22:32:47.785583 20085 raft_consensus.cc:348] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946dc4d9fc08442d85a98a7429f95240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 39005 } } peers { permanent_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41185 } } peers { permanent_uuid: "aebc573bc81347039dfc827a0b15f039" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43169 } }
I20260618 22:32:47.785918 20085 raft_consensus.cc:374] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:47.786020 20085 raft_consensus.cc:729] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aebc573bc81347039dfc827a0b15f039, State: Initialized, Role: FOLLOWER
I20260618 22:32:47.786032 20083 raft_consensus.cc:348] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946dc4d9fc08442d85a98a7429f95240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 39005 } } peers { permanent_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41185 } } peers { permanent_uuid: "aebc573bc81347039dfc827a0b15f039" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43169 } }
I20260618 22:32:47.786425 20083 raft_consensus.cc:374] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:47.786514 20083 raft_consensus.cc:729] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946dc4d9fc08442d85a98a7429f95240, State: Initialized, Role: FOLLOWER
I20260618 22:32:47.786598 20085 consensus_queue.cc:260] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946dc4d9fc08442d85a98a7429f95240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 39005 } } peers { permanent_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41185 } } peers { permanent_uuid: "aebc573bc81347039dfc827a0b15f039" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43169 } }
I20260618 22:32:47.787041 20083 consensus_queue.cc:260] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946dc4d9fc08442d85a98a7429f95240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 39005 } } peers { permanent_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41185 } } peers { permanent_uuid: "aebc573bc81347039dfc827a0b15f039" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43169 } }
I20260618 22:32:47.788151 20065 heartbeater.cc:503] Master 127.18.150.190:40407 was elected leader, sending a full tablet report...
I20260618 22:32:47.788688 20083 ts_tablet_manager.cc:1428] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260618 22:32:47.790453 20085 ts_tablet_manager.cc:1428] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260618 22:32:47.793821 20084 raft_consensus.cc:348] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946dc4d9fc08442d85a98a7429f95240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 39005 } } peers { permanent_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41185 } } peers { permanent_uuid: "aebc573bc81347039dfc827a0b15f039" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43169 } }
I20260618 22:32:47.794230 20084 raft_consensus.cc:374] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:47.794340 20084 raft_consensus.cc:729] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be8ecbea3a594237b542f9c41d5bf2ad, State: Initialized, Role: FOLLOWER
I20260618 22:32:47.794898 20084 consensus_queue.cc:260] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946dc4d9fc08442d85a98a7429f95240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 39005 } } peers { permanent_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41185 } } peers { permanent_uuid: "aebc573bc81347039dfc827a0b15f039" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43169 } }
I20260618 22:32:47.795903 19932 heartbeater.cc:503] Master 127.18.150.190:40407 was elected leader, sending a full tablet report...
I20260618 22:32:47.796448 20084 ts_tablet_manager.cc:1428] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260618 22:32:47.825999 20091 raft_consensus.cc:482] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:32:47.826275 20091 raft_consensus.cc:504] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946dc4d9fc08442d85a98a7429f95240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 39005 } } peers { permanent_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41185 } } peers { permanent_uuid: "aebc573bc81347039dfc827a0b15f039" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43169 } }
I20260618 22:32:47.827988 20091 leader_election.cc:288] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 946dc4d9fc08442d85a98a7429f95240 (127.18.150.131:39005), aebc573bc81347039dfc827a0b15f039 (127.18.150.129:43169)
I20260618 22:32:47.834626 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.828320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:38569 (local address 127.18.150.131:39005)
0618 22:32:47.828525 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.828531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.829573 (+ 1042us) server_negotiation.cc:407] Connection header received
0618 22:32:47.829722 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.829727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.829799 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.829885 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:47.830512 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.831213 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.832958 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.833715 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.833830 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.833889 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.834241 (+ 352us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.834374 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:32:47.834637 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.828490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:55407 (local address 127.18.150.129:43169)
0618 22:32:47.828682 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.828689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.829407 (+ 718us) server_negotiation.cc:407] Connection header received
0618 22:32:47.829586 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.829592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.829666 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.829746 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:47.830379 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.831251 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.833020 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.833805 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.833985 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.834056 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.834291 (+ 235us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.834413 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:32:47.834767 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.828527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:39005 (local address 127.18.150.130:38569)
0618 22:32:47.829481 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:47.829495 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:47.829618 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:47.829911 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:47.829915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:47.829992 (+ 77us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:47.830352 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.830363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.831333 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.831338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.832803 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.832817 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.833910 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.833917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.834101 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.834125 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:32:47.834195 (+ 70us) client_negotiation.cc:240] Negotiation successful
0618 22:32:47.834316 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":4,"spinlock_wait_cycles":5248,"thread_start_us":367,"threads_started":1}
I20260618 22:32:47.834789 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.828488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43169 (local address 127.18.150.130:55407)
0618 22:32:47.829270 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:47.829292 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:47.829442 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:47.829777 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:47.829781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:47.829804 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:47.830218 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.830229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.831374 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.831377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.832871 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:47.832885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.834029 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.834034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:47.834149 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.834170 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:32:47.834257 (+ 87us) client_negotiation.cc:240] Negotiation successful
0618 22:32:47.834374 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":56,"spinlock_wait_cycles":6912,"thread_start_us":676,"threads_started":1}
I20260618 22:32:47.835803 19753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac51ef3b7a64edd829bbda5b6bc0e3d" candidate_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aebc573bc81347039dfc827a0b15f039" is_pre_election: true
I20260618 22:32:47.835870 20020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac51ef3b7a64edd829bbda5b6bc0e3d" candidate_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946dc4d9fc08442d85a98a7429f95240" is_pre_election: true
I20260618 22:32:47.836410 20020 raft_consensus.cc:2454] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be8ecbea3a594237b542f9c41d5bf2ad in term 0.
I20260618 22:32:47.836426 19753 raft_consensus.cc:2454] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be8ecbea3a594237b542f9c41d5bf2ad in term 0.
I20260618 22:32:47.837097 19821 leader_election.cc:302] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 946dc4d9fc08442d85a98a7429f95240, be8ecbea3a594237b542f9c41d5bf2ad; no voters:
I20260618 22:32:47.837453 20091 raft_consensus.cc:2781] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:32:47.837590 20091 raft_consensus.cc:482] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:32:47.837685 20091 raft_consensus.cc:3037] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:47.839787 20091 raft_consensus.cc:504] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946dc4d9fc08442d85a98a7429f95240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 39005 } } peers { permanent_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41185 } } peers { permanent_uuid: "aebc573bc81347039dfc827a0b15f039" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43169 } }
I20260618 22:32:47.840363 20091 leader_election.cc:288] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [CANDIDATE]: Term 1 election: Requested vote from peers 946dc4d9fc08442d85a98a7429f95240 (127.18.150.131:39005), aebc573bc81347039dfc827a0b15f039 (127.18.150.129:43169)
I20260618 22:32:47.840758 20020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac51ef3b7a64edd829bbda5b6bc0e3d" candidate_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946dc4d9fc08442d85a98a7429f95240"
I20260618 22:32:47.840830 19753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac51ef3b7a64edd829bbda5b6bc0e3d" candidate_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aebc573bc81347039dfc827a0b15f039"
I20260618 22:32:47.840956 20020 raft_consensus.cc:3037] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:47.841006 19753 raft_consensus.cc:3037] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:47.844024 19753 raft_consensus.cc:2454] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be8ecbea3a594237b542f9c41d5bf2ad in term 1.
I20260618 22:32:47.844031 20020 raft_consensus.cc:2454] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be8ecbea3a594237b542f9c41d5bf2ad in term 1.
I20260618 22:32:47.844446 19822 leader_election.cc:302] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 946dc4d9fc08442d85a98a7429f95240, aebc573bc81347039dfc827a0b15f039, be8ecbea3a594237b542f9c41d5bf2ad; no voters:
I20260618 22:32:47.844753 20091 raft_consensus.cc:2781] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:47.845206 20091 raft_consensus.cc:686] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [term 1 LEADER]: Becoming Leader. State: Replica: be8ecbea3a594237b542f9c41d5bf2ad, State: Running, Role: LEADER
I20260618 22:32:47.845649 20091 consensus_queue.cc:237] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946dc4d9fc08442d85a98a7429f95240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 39005 } } peers { permanent_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41185 } } peers { permanent_uuid: "aebc573bc81347039dfc827a0b15f039" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43169 } }
I20260618 22:32:47.850346 19607 catalog_manager.cc:5495] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad reported cstate change: term changed from 0 to 1, leader changed from <none> to be8ecbea3a594237b542f9c41d5bf2ad (127.18.150.130). New cstate: current_term: 1 leader_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "946dc4d9fc08442d85a98a7429f95240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 39005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be8ecbea3a594237b542f9c41d5bf2ad" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aebc573bc81347039dfc827a0b15f039" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43169 } health_report { overall_health: UNKNOWN } } }
Jun 18 22:32:47 dist-test-slave-h7jz krb5kdc[19564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:32:47.880115 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.867086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47308 (local address 127.18.150.129:43169)
0618 22:32:47.867238 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.867242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.867729 (+ 487us) server_negotiation.cc:407] Connection header received
0618 22:32:47.867889 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.867894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.867958 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.868039 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:47.869429 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.870187 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.871384 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.871792 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.871899 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.875521 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:47.875535 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:47.875554 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:47.875593 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:47.878024 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.878560 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.878566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.878569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.878681 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.878939 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.878942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.878944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.879299 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:47.879504 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.879701 (+ 197us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.879850 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Jun 18 22:32:47 dist-test-slave-h7jz krb5kdc[19564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821967, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:32:47 dist-test-slave-h7jz krb5kdc[19564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821967, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
W20260618 22:32:47.916438 20066 tablet.cc:2369] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P 946dc4d9fc08442d85a98a7429f95240: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:32:47.920266 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.907518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.18.150.129:43169)
0618 22:32:47.907716 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.907722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.907992 (+ 270us) server_negotiation.cc:407] Connection header received
0618 22:32:47.908130 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.908134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.908195 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.908274 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:47.909450 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.910181 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.911355 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.911747 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.911835 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.915564 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:47.915569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:47.915573 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:47.915605 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:47.918316 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.918849 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.918853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.918855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.918984 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.919240 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.919242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.919244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.919511 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:47.919709 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.919884 (+ 175us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.920044 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Jun 18 22:32:47 dist-test-slave-h7jz krb5kdc[19564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260618 22:32:47.949045 19933 tablet.cc:2369] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P be8ecbea3a594237b542f9c41d5bf2ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 18 22:32:47 dist-test-slave-h7jz krb5kdc[19564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:32:47.958768 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.946534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.18.150.129:43169)
0618 22:32:47.946739 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.946746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.947023 (+ 277us) server_negotiation.cc:407] Connection header received
0618 22:32:47.947164 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.947168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.947228 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.947310 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:47.948539 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.949214 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.950592 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.951001 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.951088 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.954599 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:47.954604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:47.954608 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:47.954638 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:47.956740 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.957240 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.957244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.957246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.957356 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.957640 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.957642 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.957644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.957898 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:47.958161 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.958352 (+ 191us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.958527 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260618 22:32:47.960368 19713 tablet_service.cc:2908] Rejecting scan request for tablet 7ac51ef3b7a64edd829bbda5b6bc0e3d: Uninitialized: clean time has not yet been initialized
I20260618 22:32:47.962934 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 19670
I20260618 22:32:47.973749 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:47.963755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47338 (local address 127.18.150.129:43169)
0618 22:32:47.963893 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:47.963898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:47.964276 (+ 378us) server_negotiation.cc:407] Connection header received
0618 22:32:47.964391 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:47.964394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:47.964460 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:47.964535 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:47.965560 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.966289 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.967352 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:47.967735 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:47.967831 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:47.969731 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:47.969735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:47.969738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:47.969763 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:47.971896 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.972436 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.972440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.972442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.972542 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:47.972782 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:47.972785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:47.972787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:47.973034 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:47.973231 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:47.973384 (+ 153us) server_negotiation.cc:299] Negotiation successful
0618 22:32:47.973536 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:32:47.974370 19793 generic_service.cc:196] Checking for leaks (request via RPC)
W20260618 22:32:48.059926 19799 tablet.cc:2369] T 7ac51ef3b7a64edd829bbda5b6bc0e3d P aebc573bc81347039dfc827a0b15f039: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:32:48.060659 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 19670
I20260618 22:32:48.073482 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 19804
Jun 18 22:32:48 dist-test-slave-h7jz krb5kdc[19564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:32:48.087132 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:48.074194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58106 (local address 127.18.150.130:41185)
0618 22:32:48.074382 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:48.074388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:48.074408 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:32:48.074594 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:48.074598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:48.074677 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:48.074765 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:48.075801 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:48.076488 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:48.077623 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:48.078130 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:48.078223 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:48.081796 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:48.081810 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:48.081826 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:48.081859 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:48.084727 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:48.085404 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:48.085411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:48.085414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:48.085558 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:48.085838 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:48.085841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:48.085843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:48.086261 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:48.086467 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:48.086687 (+ 220us) server_negotiation.cc:299] Negotiation successful
0618 22:32:48.086874 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:32:48.087780 19927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:48.173722 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 19804
I20260618 22:32:48.186041 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 19937
Jun 18 22:32:48 dist-test-slave-h7jz krb5kdc[19564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:32:48.199683 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:48.186770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50160 (local address 127.18.150.131:39005)
0618 22:32:48.186924 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:48.186929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:48.186965 (+ 36us) server_negotiation.cc:407] Connection header received
0618 22:32:48.187166 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:48.187170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:48.187230 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:48.187321 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:48.188534 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:48.189231 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:48.190521 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:48.190903 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:48.191005 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:48.194854 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:48.194868 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:48.194884 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:48.194915 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:48.197467 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:48.198030 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:48.198036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:48.198039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:48.198158 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:48.198509 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:48.198512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:48.198514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:48.198857 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:48.199057 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:48.199285 (+ 228us) server_negotiation.cc:299] Negotiation successful
0618 22:32:48.199435 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:32:48.200274 20060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:48.284540 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 19937
I20260618 22:32:48.296957 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 19577
I20260618 22:32:48.298071 19639 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:48.401521 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 19577
2026-06-18T22:32:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3813 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:48 dist-test-slave-h7jz krb5kdc[20113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:48 dist-test-slave-h7jz krb5kdc[20113](info): set up 2 sockets
Jun 18 22:32:48 dist-test-slave-h7jz krb5kdc[20113](info): commencing operation
krb5kdc: starting...
W20260618 22:32:49.385708 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.003s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:32:49 dist-test-slave-h7jz krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:32:49.551321 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35947
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:35947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:49.861261 20126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:49.861555 20126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:49.861646 20126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:49.868930 20126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:49.869031 20126 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260618 22:32:49.869073 20126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:49.869131 20126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:49.869179 20126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:49.869235 20126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:49.878924 20126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:35947
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:49.880131 20126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:49.881371 20126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:49.892446 20131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:49.892508 20132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:49.892668 20134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:49.893077 20126 server_base.cc:1034] running on GCE node
I20260618 22:32:49.893744 20126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:49.895093 20126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:49.896306 20126 hybrid_clock.cc:648] HybridClock initialized: now 1781821969896248 us; error 33 us; skew 500 ppm
I20260618 22:32:49.896585 20126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:49 dist-test-slave-h7jz krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821969, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:49.899381 20126 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:32:49.900671 20126 webserver.cc:460] Webserver started at http://127.18.150.190:36747/ using document root <none> and password file <none>
I20260618 22:32:49.901310 20126 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:49.901389 20126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:49.901639 20126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:49.904422 20126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "ab21c91869ac4c83bbbc7329724442c6"
format_stamp: "Formatted at 2026-06-18 22:32:49 on dist-test-slave-h7jz"
server_key: "2f067854cd5037ed7cdbb933d398bb49"
server_key_iv: "23e2314769a046251716d11f908ae1d3"
server_key_version: "encryptionkey@0"
I20260618 22:32:49.905097 20126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "ab21c91869ac4c83bbbc7329724442c6"
format_stamp: "Formatted at 2026-06-18 22:32:49 on dist-test-slave-h7jz"
server_key: "2f067854cd5037ed7cdbb933d398bb49"
server_key_iv: "23e2314769a046251716d11f908ae1d3"
server_key_version: "encryptionkey@0"
I20260618 22:32:49.909843 20126 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260618 22:32:49.913000 20141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:49.914274 20126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:32:49.914407 20126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "ab21c91869ac4c83bbbc7329724442c6"
format_stamp: "Formatted at 2026-06-18 22:32:49 on dist-test-slave-h7jz"
server_key: "2f067854cd5037ed7cdbb933d398bb49"
server_key_iv: "23e2314769a046251716d11f908ae1d3"
server_key_version: "encryptionkey@0"
I20260618 22:32:49.914577 20126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:49.941061 20126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:49.944778 20126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:49.944988 20126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:49.964058 20126 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:35947
I20260618 22:32:49.964071 20202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:35947 every 8 connection(s)
I20260618 22:32:49.965379 20126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:49.966396 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 20126
I20260618 22:32:49.966624 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:32:49.967408 19034 external_mini_cluster.cc:1428] Setting key 052c527ee77a1dc756f19319f9b29163
I20260618 22:32:49.970036 20203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 18 22:32:49 dist-test-slave-h7jz krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:49.982372 20203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6: Bootstrap starting.
I20260618 22:32:49.985683 20203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:49.986819 20203 log.cc:824] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:49.989686 20206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:49.969422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49926 (local address 127.18.150.190:35947)
0618 22:32:49.970216 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:49.970228 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:49.970268 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:32:49.971819 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:49.971840 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:49.972231 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:49.972577 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:49.974221 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:49.975582 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:49.977274 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:49.977996 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:49.978145 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:49.982599 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:49.982607 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:49.982623 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:49.982666 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:49.985497 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:49.986370 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:49.986378 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:49.986384 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:49.986543 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:49.986908 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:49.986911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:49.986913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:49.987602 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:49.987931 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:49.988263 (+ 332us) server_negotiation.cc:299] Negotiation successful
0618 22:32:49.988684 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":372,"threads_started":1}
I20260618 22:32:49.989858 20203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6: No bootstrap required, opened a new log
I20260618 22:32:49.996965 20203 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab21c91869ac4c83bbbc7329724442c6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35947 } }
I20260618 22:32:49.997339 20203 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:49.997429 20203 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab21c91869ac4c83bbbc7329724442c6, State: Initialized, Role: FOLLOWER
I20260618 22:32:49.998005 20203 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab21c91869ac4c83bbbc7329724442c6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35947 } }
I20260618 22:32:49.998219 20203 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:49.998334 20203 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:49.998484 20203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:50.000522 20203 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab21c91869ac4c83bbbc7329724442c6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35947 } }
I20260618 22:32:50.000980 20203 leader_election.cc:302] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab21c91869ac4c83bbbc7329724442c6; no voters:
I20260618 22:32:50.001490 20203 leader_election.cc:288] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:50.001610 20208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:50.002276 20208 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [term 1 LEADER]: Becoming Leader. State: Replica: ab21c91869ac4c83bbbc7329724442c6, State: Running, Role: LEADER
I20260618 22:32:50.002769 20208 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab21c91869ac4c83bbbc7329724442c6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35947 } }
I20260618 22:32:50.003414 20203 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:50.005317 20210 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab21c91869ac4c83bbbc7329724442c6. Latest consensus state: current_term: 1 leader_uuid: "ab21c91869ac4c83bbbc7329724442c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab21c91869ac4c83bbbc7329724442c6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35947 } } }
I20260618 22:32:50.005713 20210 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:50.006502 20216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:50.006376 20209 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab21c91869ac4c83bbbc7329724442c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab21c91869ac4c83bbbc7329724442c6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35947 } } }
I20260618 22:32:50.007624 20209 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:50.011942 20216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:50.020548 20216 catalog_manager.cc:1347] Generated new cluster ID: fe406ba14fdc437bb4798cec7d7fbcd7
I20260618 22:32:50.020658 20216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:50.033896 20216 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:50.035204 20216 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:50.048267 20216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6: Generated new TSK 0
I20260618 22:32:50.049125 20216 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:32:50.097019 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35947
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:50.412591 20229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:50.412840 20229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:50.412935 20229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:50.419946 20229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:50.420045 20229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:50.420132 20229 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:32:50.420213 20229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:32:50.429487 20229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.150.190:35947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:50.430693 20229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:50.431921 20229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:50.443996 20234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:50.444360 20237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:50.444340 20235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:50.445235 20229 server_base.cc:1034] running on GCE node
I20260618 22:32:50.445822 20229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:50.446727 20229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:50.447950 20229 hybrid_clock.cc:648] HybridClock initialized: now 1781821970447876 us; error 45 us; skew 500 ppm
I20260618 22:32:50.448243 20229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:50 dist-test-slave-h7jz krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821970, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:50.451201 20229 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:32:50.452581 20229 webserver.cc:460] Webserver started at http://127.18.150.129:46601/ using document root <none> and password file <none>
I20260618 22:32:50.453249 20229 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:50.453341 20229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:50.453622 20229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:50.456386 20229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "e591116bfc9b4d9b9f53919715edab09"
format_stamp: "Formatted at 2026-06-18 22:32:50 on dist-test-slave-h7jz"
server_key: "c8e6126b57d7bbdfb3b1a3aa32ed78eb"
server_key_iv: "028bf9c7491abf2dab73560ab462b4f6"
server_key_version: "encryptionkey@0"
I20260618 22:32:50.457129 20229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "e591116bfc9b4d9b9f53919715edab09"
format_stamp: "Formatted at 2026-06-18 22:32:50 on dist-test-slave-h7jz"
server_key: "c8e6126b57d7bbdfb3b1a3aa32ed78eb"
server_key_iv: "028bf9c7491abf2dab73560ab462b4f6"
server_key_version: "encryptionkey@0"
I20260618 22:32:50.462005 20229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260618 22:32:50.465286 20244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:50.466600 20229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:32:50.466738 20229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "e591116bfc9b4d9b9f53919715edab09"
format_stamp: "Formatted at 2026-06-18 22:32:50 on dist-test-slave-h7jz"
server_key: "c8e6126b57d7bbdfb3b1a3aa32ed78eb"
server_key_iv: "028bf9c7491abf2dab73560ab462b4f6"
server_key_version: "encryptionkey@0"
I20260618 22:32:50.466914 20229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:50.499560 20229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:50.503765 20229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:50.503991 20229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:50.506592 20229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:50.506688 20229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:50.506810 20229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:50.506886 20229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:32:50 dist-test-slave-h7jz krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821970, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:50.528838 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:50.507186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:43975 (local address 127.18.150.190:35947)
0618 22:32:50.509648 (+ 2462us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:50.509654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:50.509677 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:32:50.511177 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:50.511183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:50.511270 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:50.511384 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:50.513758 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:50.514795 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:50.517234 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:50.517986 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:50.518125 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:50.521888 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:50.521895 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:50.521901 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:50.521963 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:50.525116 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:50.525792 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:50.525797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:50.525801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:50.525973 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:50.526520 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:50.526525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:50.526528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:50.526829 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:50.528006 (+ 1177us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:50.528137 (+ 131us) server_negotiation.cc:299] Negotiation successful
0618 22:32:50.528385 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2349,"thread_start_us":2245,"threads_started":1}
I20260618 22:32:50.529639 20254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:50.507941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35947 (local address 127.18.150.129:43975)
0618 22:32:50.508933 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:50.508992 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:50.510905 (+ 1913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:50.511814 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:50.511829 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:50.512637 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:50.513550 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:50.513583 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:50.515035 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:50.515041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:50.517043 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:50.517062 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:50.518229 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:50.518235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:50.518424 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:50.519171 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:50.519193 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:50.521622 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:50.525257 (+ 3635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:50.525266 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:50.525284 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:50.525681 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:50.526110 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:50.526114 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:50.526118 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:50.526350 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:50.526990 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:50.526999 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:50.527460 (+ 461us) client_negotiation.cc:769] Sending connection context
0618 22:32:50.527771 (+ 311us) client_negotiation.cc:240] Negotiation successful
0618 22:32:50.528299 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":403,"threads_started":1}
I20260618 22:32:50.549124 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:50.535251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:36399 (local address 127.18.150.190:35947)
0618 22:32:50.535444 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:50.535450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:50.535935 (+ 485us) server_negotiation.cc:407] Connection header received
0618 22:32:50.536066 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:50.536071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:50.536149 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:50.536256 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:50.537098 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:50.538261 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:50.540192 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:50.540897 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:50.541018 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:50.543267 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:50.543272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:50.543275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:50.543318 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:50.546447 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:50.547312 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:50.547316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:50.547318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:50.547446 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:50.547872 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:50.547884 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:50.547886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:50.548143 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:50.548245 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:50.548732 (+ 487us) server_negotiation.cc:299] Negotiation successful
0618 22:32:50.548895 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:32:50.549444 20323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:50.535276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35947 (local address 127.18.150.129:36399)
0618 22:32:50.535846 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:50.535864 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:50.535973 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:50.536274 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:50.536277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:50.536710 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:50.536980 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:50.536988 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:50.538486 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:50.538491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:50.540039 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:50.540054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:50.541061 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:50.541065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:50.541168 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:50.541849 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:50.541875 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:50.543021 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:50.546607 (+ 3586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:50.546613 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:50.546617 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:50.547151 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:50.547569 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:50.547577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:50.547580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:50.547733 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:50.548258 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:50.548264 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:50.548473 (+ 209us) client_negotiation.cc:769] Sending connection context
0618 22:32:50.548633 (+ 160us) client_negotiation.cc:240] Negotiation successful
0618 22:32:50.548969 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":296,"threads_started":1}
I20260618 22:32:50.558120 20229 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:35183
I20260618 22:32:50.558138 20364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:35183 every 8 connection(s)
I20260618 22:32:50.559262 20229 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:32:50.561285 20365 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35947
I20260618 22:32:50.561501 20365 heartbeater.cc:464] Registering TS with master...
I20260618 22:32:50.562166 20365 heartbeater.cc:511] Master 127.18.150.190:35947 requested a full tablet report, sending...
I20260618 22:32:50.564025 20158 ts_manager.cc:194] Registered new tserver with Master: e591116bfc9b4d9b9f53919715edab09 (127.18.150.129:35183)
I20260618 22:32:50.564833 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 20229
I20260618 22:32:50.565017 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:32:50.565542 20158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:43975
I20260618 22:32:50.565609 19034 external_mini_cluster.cc:1428] Setting key e2cc38417dfd91f5999b898018c752c1
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:32:50.601708 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35947
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:50.908329 20369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:50.908620 20369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:50.908731 20369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:50.915918 20369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:50.916036 20369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:50.916133 20369 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:32:50.916217 20369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:32:50.926466 20369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.150.190:35947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:50.927700 20369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:50.928963 20369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:50.941176 20374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:50.941236 20375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:50.941211 20377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:50.942067 20369 server_base.cc:1034] running on GCE node
I20260618 22:32:50.942606 20369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:50.943475 20369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:50.944672 20369 hybrid_clock.cc:648] HybridClock initialized: now 1781821970944607 us; error 53 us; skew 500 ppm
I20260618 22:32:50.944953 20369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:50 dist-test-slave-h7jz krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821970, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:50.948011 20369 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:32:50.949414 20369 webserver.cc:460] Webserver started at http://127.18.150.130:39173/ using document root <none> and password file <none>
I20260618 22:32:50.950117 20369 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:50.950205 20369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:50.950527 20369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:50.953197 20369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "394ea4e8bbf54383b5e46f41d30ba49e"
format_stamp: "Formatted at 2026-06-18 22:32:50 on dist-test-slave-h7jz"
server_key: "0a1135fa6ee95675de4133127b6b1d6f"
server_key_iv: "448f31acc5d412ca01536e3ceff6307d"
server_key_version: "encryptionkey@0"
I20260618 22:32:50.953909 20369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "394ea4e8bbf54383b5e46f41d30ba49e"
format_stamp: "Formatted at 2026-06-18 22:32:50 on dist-test-slave-h7jz"
server_key: "0a1135fa6ee95675de4133127b6b1d6f"
server_key_iv: "448f31acc5d412ca01536e3ceff6307d"
server_key_version: "encryptionkey@0"
I20260618 22:32:50.958765 20369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:32:50.961921 20384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:50.963258 20369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:32:50.963397 20369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "394ea4e8bbf54383b5e46f41d30ba49e"
format_stamp: "Formatted at 2026-06-18 22:32:50 on dist-test-slave-h7jz"
server_key: "0a1135fa6ee95675de4133127b6b1d6f"
server_key_iv: "448f31acc5d412ca01536e3ceff6307d"
server_key_version: "encryptionkey@0"
I20260618 22:32:50.963584 20369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:51.019414 20369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:51.023509 20369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:51.023772 20369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:51.026394 20369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:51.026556 20369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260618 22:32:51.026675 20369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:51.026736 20369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:32:51 dist-test-slave-h7jz krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821970, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:51.050957 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.027156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:47049 (local address 127.18.150.190:35947)
0618 22:32:51.027342 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.027349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.028777 (+ 1428us) server_negotiation.cc:407] Connection header received
0618 22:32:51.031603 (+ 2826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.031610 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.031698 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.031775 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:51.034687 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.035577 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.038478 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.039022 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.039168 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.043993 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:51.044000 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:51.044005 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:51.044048 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:51.047381 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.048409 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.048414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.048417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.048598 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.049088 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.049091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.049093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.049343 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:51.049495 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.050520 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.050689 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:32:51.052531 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.027747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35947 (local address 127.18.150.130:47049)
0618 22:32:51.028588 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:51.028639 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:51.031178 (+ 2539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:51.032412 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:51.032429 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:51.033392 (+ 963us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:51.034466 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.034497 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.035772 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.035777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.038288 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.038303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.039283 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.039294 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.039624 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.040459 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:51.040494 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:51.043735 (+ 3241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:51.047543 (+ 3808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:51.047554 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:51.047577 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:51.048237 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:51.048721 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:51.048727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:51.048731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:51.048986 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:51.049495 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:51.049517 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:51.050030 (+ 513us) client_negotiation.cc:769] Sending connection context
0618 22:32:51.050384 (+ 354us) client_negotiation.cc:240] Negotiation successful
0618 22:32:51.050964 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":336,"threads_started":1}
I20260618 22:32:51.072655 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.057981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:40493 (local address 127.18.150.190:35947)
0618 22:32:51.058167 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.058172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.059003 (+ 831us) server_negotiation.cc:407] Connection header received
0618 22:32:51.059182 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.059185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.059243 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.059319 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:51.060471 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.061261 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.063363 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.063816 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.063913 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.066297 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:51.066303 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:51.066307 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:51.066347 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:51.069589 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.070472 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.070476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.070478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.070671 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.071195 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.071199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.071201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.071555 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:51.071701 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.072260 (+ 559us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.072434 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:32:51.072945 20471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.058068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35947 (local address 127.18.150.130:40493)
0618 22:32:51.058887 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:51.058909 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:51.059072 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:51.059359 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:51.059364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:51.059934 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:51.060327 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.060339 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.061387 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.061393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.063205 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.063224 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.064000 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.064007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.064220 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.064922 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:51.064942 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:51.066048 (+ 1106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:51.069740 (+ 3692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:51.069744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:51.069750 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:51.070326 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:51.070819 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:51.070823 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:51.070827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:51.071005 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:51.071697 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:51.071702 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:51.071937 (+ 235us) client_negotiation.cc:769] Sending connection context
0618 22:32:51.072123 (+ 186us) client_negotiation.cc:240] Negotiation successful
0618 22:32:51.072498 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":400,"threads_started":1}
I20260618 22:32:51.074390 20369 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:42641
I20260618 22:32:51.074409 20503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:42641 every 8 connection(s)
I20260618 22:32:51.075729 20369 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:32:51.077672 20504 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35947
I20260618 22:32:51.077682 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 20369
I20260618 22:32:51.077839 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:32:51.077869 20504 heartbeater.cc:464] Registering TS with master...
I20260618 22:32:51.078264 19034 external_mini_cluster.cc:1428] Setting key 203b1fd044c37c5ff46b193851413745
I20260618 22:32:51.078509 20504 heartbeater.cc:511] Master 127.18.150.190:35947 requested a full tablet report, sending...
I20260618 22:32:51.079725 20157 ts_manager.cc:194] Registered new tserver with Master: 394ea4e8bbf54383b5e46f41d30ba49e (127.18.150.130:42641)
I20260618 22:32:51.080524 20157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:47049
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:32:51.113413 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35947
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:45057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:51.412108 20508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:51.412391 20508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:51.412521 20508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:51.419613 20508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:51.419734 20508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:51.419854 20508 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:32:51.419961 20508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:32:51.429512 20508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:45057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.150.190:35947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:51.430781 20508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:51.432111 20508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:51.444325 20513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:51.444389 20516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:51.444360 20514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:51.445101 20508 server_base.cc:1034] running on GCE node
I20260618 22:32:51.445734 20508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:51.446718 20508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:51.447976 20508 hybrid_clock.cc:648] HybridClock initialized: now 1781821971447899 us; error 38 us; skew 500 ppm
I20260618 22:32:51.448294 20508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:51 dist-test-slave-h7jz krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821971, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:51.454650 20508 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:32:51.456151 20508 webserver.cc:460] Webserver started at http://127.18.150.131:34913/ using document root <none> and password file <none>
I20260618 22:32:51.456849 20508 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:51.456945 20508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:51.457238 20508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:51.460116 20508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "739e0782e21440c8904811a075c91e0f"
format_stamp: "Formatted at 2026-06-18 22:32:51 on dist-test-slave-h7jz"
server_key: "a93a967748a24ebda576fb5db1fc2f0f"
server_key_iv: "9c01fdaedba35db1cf73dc0bd5fc2fce"
server_key_version: "encryptionkey@0"
I20260618 22:32:51.460877 20508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "739e0782e21440c8904811a075c91e0f"
format_stamp: "Formatted at 2026-06-18 22:32:51 on dist-test-slave-h7jz"
server_key: "a93a967748a24ebda576fb5db1fc2f0f"
server_key_iv: "9c01fdaedba35db1cf73dc0bd5fc2fce"
server_key_version: "encryptionkey@0"
I20260618 22:32:51.465987 20508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260618 22:32:51.469381 20523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:51.470747 20508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:32:51.470888 20508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "739e0782e21440c8904811a075c91e0f"
format_stamp: "Formatted at 2026-06-18 22:32:51 on dist-test-slave-h7jz"
server_key: "a93a967748a24ebda576fb5db1fc2f0f"
server_key_iv: "9c01fdaedba35db1cf73dc0bd5fc2fce"
server_key_version: "encryptionkey@0"
I20260618 22:32:51.471119 20508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:51.484130 20508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:51.488063 20508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:51.488293 20508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:51.490782 20508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:51.490861 20508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:51.490988 20508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:51.491046 20508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:32:51 dist-test-slave-h7jz krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821971, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:51.512063 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.491300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:43987 (local address 127.18.150.190:35947)
0618 22:32:51.491479 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.491485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.492804 (+ 1319us) server_negotiation.cc:407] Connection header received
0618 22:32:51.494787 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.494793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.494879 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.494996 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:51.497181 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.498366 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.500591 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.501134 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.501246 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.506040 (+ 4794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:51.506048 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:51.506053 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:51.506104 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:51.508727 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.509409 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.509415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.509419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.509540 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.509957 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.509962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.509966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.510258 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:51.510404 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.511466 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.511708 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260618 22:32:51.513931 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.491808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35947 (local address 127.18.150.131:43987)
0618 22:32:51.492615 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:51.492668 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:51.494475 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:51.495383 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:51.495392 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:51.496099 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:51.496902 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.496923 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.498534 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.498540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.500425 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.500462 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.501359 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.501366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.501631 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.502407 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:51.502443 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:51.505787 (+ 3344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:51.508852 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:51.508861 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:51.508878 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:51.509299 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:51.509638 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:51.509643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:51.509647 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:51.509838 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:51.510406 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:51.510418 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:51.510976 (+ 558us) client_negotiation.cc:769] Sending connection context
0618 22:32:51.511363 (+ 387us) client_negotiation.cc:240] Negotiation successful
0618 22:32:51.512010 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":334,"threads_started":1}
I20260618 22:32:51.532692 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.519342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:47049 (local address 127.18.150.190:35947)
0618 22:32:51.519490 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.519494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.520052 (+ 558us) server_negotiation.cc:407] Connection header received
0618 22:32:51.520218 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.520223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.520299 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.520377 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:51.521264 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.522001 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.523935 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.524581 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.524716 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.526536 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:51.526542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:51.526546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:51.526590 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:51.529933 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.530737 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.530742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.530746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.530863 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.531283 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.531288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.531292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.531636 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:51.531778 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.532177 (+ 399us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.532336 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:32:51.532984 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.519426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35947 (local address 127.18.150.131:47049)
0618 22:32:51.519954 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:51.519973 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:51.520109 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:51.520411 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:51.520416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:51.520861 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:51.521134 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.521143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.522149 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.522154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.523727 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.523742 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.524785 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.524792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.524907 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.525516 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:51.525534 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:51.526370 (+ 836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:51.530098 (+ 3728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:51.530103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:51.530107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:51.530587 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:51.530996 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:51.531001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:51.531005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:51.531183 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:51.531770 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:51.531775 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:51.531958 (+ 183us) client_negotiation.cc:769] Sending connection context
0618 22:32:51.532116 (+ 158us) client_negotiation.cc:240] Negotiation successful
0618 22:32:51.532478 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":282,"threads_started":1}
I20260618 22:32:51.538805 20508 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:38939
I20260618 22:32:51.538836 20642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:38939 every 8 connection(s)
I20260618 22:32:51.540145 20508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:32:51.541884 20643 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35947
I20260618 22:32:51.542096 20643 heartbeater.cc:464] Registering TS with master...
I20260618 22:32:51.542672 20643 heartbeater.cc:511] Master 127.18.150.190:35947 requested a full tablet report, sending...
I20260618 22:32:51.543797 20157 ts_manager.cc:194] Registered new tserver with Master: 739e0782e21440c8904811a075c91e0f (127.18.150.131:38939)
I20260618 22:32:51.544603 20157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:43987
I20260618 22:32:51.548398 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 20508
I20260618 22:32:51.548550 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:32:51.548939 19034 external_mini_cluster.cc:1428] Setting key 8310bc5d628864978f5cd1779bd60525
I20260618 22:32:51.551529 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:32:51.568639 20365 heartbeater.cc:503] Master 127.18.150.190:35947 was elected leader, sending a full tablet report...
Jun 18 22:32:51 dist-test-slave-h7jz krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821971, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:32:51 dist-test-slave-h7jz krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821971, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:51.591786 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.579111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49938 (local address 127.18.150.190:35947)
0618 22:32:51.579284 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.579290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.579607 (+ 317us) server_negotiation.cc:407] Connection header received
0618 22:32:51.579762 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.579766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.579822 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.579901 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:51.581156 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.581870 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.583085 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.583508 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.583618 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.587109 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:51.587113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:51.587116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:51.587148 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:51.589415 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.589999 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.590005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.590008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.590127 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.590454 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.590459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.590462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.590724 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:51.590937 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.591262 (+ 325us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.591467 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:32:51.595464 20157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:32:51.602746 20157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:32:51.641062 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.614243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:38939 (local address 127.0.0.1:59898)
0618 22:32:51.631804 (+ 17561us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:51.631823 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:51.631940 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:51.633833 (+ 1893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:51.633843 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:51.633877 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:51.634771 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.634788 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.636468 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.636472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.638467 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.638482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.640085 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.640091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.640270 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.640318 (+ 48us) client_negotiation.cc:769] Sending connection context
0618 22:32:51.640437 (+ 119us) client_negotiation.cc:240] Negotiation successful
0618 22:32:51.640622 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17433,"mutex_wait_us":5,"spinlock_wait_cycles":1792,"thread_start_us":20349,"threads_started":1}
I20260618 22:32:51.641904 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.614509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59898 (local address 127.18.150.131:38939)
0618 22:32:51.633080 (+ 18571us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.633089 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.633127 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:32:51.633242 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.633251 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.633567 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.634172 (+ 605us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:51.635021 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.636358 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.638616 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.639904 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.640084 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.640256 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.641436 (+ 1180us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.641615 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18388,"thread_start_us":18204,"threads_started":1}
I20260618 22:32:51.642127 20658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.614233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51908 (local address 127.18.150.130:42641)
0618 22:32:51.633336 (+ 19103us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.633344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.633373 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:32:51.633468 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.633476 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.633716 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.634987 (+ 1271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:51.635140 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.636360 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.638955 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.640259 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.640418 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.640569 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.641679 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.641839 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18944,"thread_start_us":18829,"threads_started":1}
I20260618 22:32:51.641544 20656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.614081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:42641 (local address 127.0.0.1:51908)
0618 22:32:51.629071 (+ 14990us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:51.629127 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:51.629358 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:51.633924 (+ 4566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:51.633928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:51.633989 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:51.634775 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.634788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.636507 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.636516 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.638790 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.638803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.640778 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.640787 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.640940 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.640971 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:32:51.641083 (+ 112us) client_negotiation.cc:240] Negotiation successful
0618 22:32:51.641201 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14653,"mutex_wait_us":15,"thread_start_us":15501,"threads_started":1}
I20260618 22:32:51.642562 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.616711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59448 (local address 127.18.150.129:35183)
0618 22:32:51.632526 (+ 15815us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.632536 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.634978 (+ 2442us) server_negotiation.cc:407] Connection header received
0618 22:32:51.635250 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.635258 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.635472 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.635662 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:51.636187 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.637778 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.640159 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.641256 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.641387 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.641549 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.642101 (+ 552us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.642274 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15565,"thread_start_us":15420,"threads_started":1}
I20260618 22:32:51.643113 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.616133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:35183 (local address 127.0.0.1:59448)
0618 22:32:51.634875 (+ 18742us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:51.634890 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:51.635070 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:51.635672 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:51.635677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:51.635700 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:51.636032 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.636043 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.637925 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.637930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.639987 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.640000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.641721 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.641727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.641890 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.641919 (+ 29us) client_negotiation.cc:769] Sending connection context
0618 22:32:51.642779 (+ 860us) client_negotiation.cc:240] Negotiation successful
0618 22:32:51.642869 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18621,"mutex_wait_us":21,"thread_start_us":15369,"threads_started":1}
I20260618 22:32:51.645982 20434 tablet_service.cc:1467] Processing CreateTablet for tablet 8e45121b9bc949eba8f6fc309e64ce54 (DEFAULT_TABLE table=test-table [id=b449ed3e89a84507b938598967f58352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:51.645991 20573 tablet_service.cc:1467] Processing CreateTablet for tablet 8e45121b9bc949eba8f6fc309e64ce54 (DEFAULT_TABLE table=test-table [id=b449ed3e89a84507b938598967f58352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:51.645906 20295 tablet_service.cc:1467] Processing CreateTablet for tablet 8e45121b9bc949eba8f6fc309e64ce54 (DEFAULT_TABLE table=test-table [id=b449ed3e89a84507b938598967f58352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:51.647264 20295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e45121b9bc949eba8f6fc309e64ce54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:51.647305 20434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e45121b9bc949eba8f6fc309e64ce54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:51.647303 20573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e45121b9bc949eba8f6fc309e64ce54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:51.657759 20662 tablet_bootstrap.cc:492] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09: Bootstrap starting.
I20260618 22:32:51.661280 20662 tablet_bootstrap.cc:654] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:51.662114 20663 tablet_bootstrap.cc:492] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e: Bootstrap starting.
I20260618 22:32:51.662572 20662 log.cc:824] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:51.662858 20664 tablet_bootstrap.cc:492] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f: Bootstrap starting.
I20260618 22:32:51.665184 20662 tablet_bootstrap.cc:492] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09: No bootstrap required, opened a new log
I20260618 22:32:51.665424 20662 ts_tablet_manager.cc:1397] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260618 22:32:51.666343 20663 tablet_bootstrap.cc:654] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:51.666352 20664 tablet_bootstrap.cc:654] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:51.667541 20664 log.cc:824] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:51.667580 20663 log.cc:824] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:51.671065 20663 tablet_bootstrap.cc:492] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e: No bootstrap required, opened a new log
I20260618 22:32:51.671300 20663 ts_tablet_manager.cc:1397] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260618 22:32:51.671314 20664 tablet_bootstrap.cc:492] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f: No bootstrap required, opened a new log
I20260618 22:32:51.671576 20664 ts_tablet_manager.cc:1397] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260618 22:32:51.674626 20662 raft_consensus.cc:348] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.675082 20662 raft_consensus.cc:374] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:51.675186 20662 raft_consensus.cc:729] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e591116bfc9b4d9b9f53919715edab09, State: Initialized, Role: FOLLOWER
I20260618 22:32:51.675806 20662 consensus_queue.cc:260] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.677526 20662 ts_tablet_manager.cc:1428] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260618 22:32:51.680012 20663 raft_consensus.cc:348] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.680437 20663 raft_consensus.cc:374] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:51.680549 20663 raft_consensus.cc:729] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394ea4e8bbf54383b5e46f41d30ba49e, State: Initialized, Role: FOLLOWER
I20260618 22:32:51.680850 20664 raft_consensus.cc:348] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.681228 20664 raft_consensus.cc:374] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:51.681084 20663 consensus_queue.cc:260] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.681324 20664 raft_consensus.cc:729] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739e0782e21440c8904811a075c91e0f, State: Initialized, Role: FOLLOWER
I20260618 22:32:51.681918 20664 consensus_queue.cc:260] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.682075 20504 heartbeater.cc:503] Master 127.18.150.190:35947 was elected leader, sending a full tablet report...
I20260618 22:32:51.683272 20643 heartbeater.cc:503] Master 127.18.150.190:35947 was elected leader, sending a full tablet report...
I20260618 22:32:51.683399 20664 ts_tablet_manager.cc:1428] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
I20260618 22:32:51.685024 20663 ts_tablet_manager.cc:1428] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e: Time spent starting tablet: real 0.014s user 0.008s sys 0.003s
I20260618 22:32:51.702198 20670 raft_consensus.cc:482] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:32:51.702451 20670 raft_consensus.cc:504] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.704056 20670 leader_election.cc:288] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 394ea4e8bbf54383b5e46f41d30ba49e (127.18.150.130:42641), e591116bfc9b4d9b9f53919715edab09 (127.18.150.129:35183)
I20260618 22:32:51.711185 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.704631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:53297 (local address 127.18.150.129:35183)
0618 22:32:51.704838 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.704844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.705195 (+ 351us) server_negotiation.cc:407] Connection header received
0618 22:32:51.705313 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.705317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.705379 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.705443 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:51.705911 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.706830 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.709075 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.710241 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.710368 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.710429 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.710816 (+ 387us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.710946 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:32:51.711189 20658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.704319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:60537 (local address 127.18.150.130:42641)
0618 22:32:51.704527 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.704533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.704554 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:32:51.704939 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.704944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.705027 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.705098 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:51.705552 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.706563 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.708974 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.710160 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.710317 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.710389 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.710831 (+ 442us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.710951 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:32:51.711380 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.704325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:35183 (local address 127.18.150.131:53297)
0618 22:32:51.705096 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:51.705112 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:51.705213 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:51.705485 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:51.705490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:51.705515 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:51.705773 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.705780 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.706965 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.706970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.708944 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.708955 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.710435 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.710440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.710590 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.710689 (+ 99us) client_negotiation.cc:769] Sending connection context
0618 22:32:51.710781 (+ 92us) client_negotiation.cc:240] Negotiation successful
0618 22:32:51.711001 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":317,"thread_start_us":269,"threads_started":1}
I20260618 22:32:51.711915 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.704303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:42641 (local address 127.18.150.131:60537)
0618 22:32:51.704485 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:51.704505 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:51.704620 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:51.705144 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:51.705150 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:51.705177 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:51.705415 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.705425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.706741 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.706747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.708828 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.708845 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.710347 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.710353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.710594 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.710685 (+ 91us) client_negotiation.cc:769] Sending connection context
0618 22:32:51.710781 (+ 96us) client_negotiation.cc:240] Negotiation successful
0618 22:32:51.711001 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":93,"spinlock_wait_cycles":22272}
I20260618 22:32:51.712328 20319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e45121b9bc949eba8f6fc309e64ce54" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e591116bfc9b4d9b9f53919715edab09" is_pre_election: true
I20260618 22:32:51.712771 20319 raft_consensus.cc:2454] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 0.
I20260618 22:32:51.712716 20454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e45121b9bc949eba8f6fc309e64ce54" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" is_pre_election: true
I20260618 22:32:51.713125 20454 raft_consensus.cc:2454] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 0.
I20260618 22:32:51.713387 20525 leader_election.cc:302] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 739e0782e21440c8904811a075c91e0f, e591116bfc9b4d9b9f53919715edab09; no voters:
I20260618 22:32:51.713783 20670 raft_consensus.cc:2781] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:32:51.713907 20670 raft_consensus.cc:482] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:32:51.714020 20670 raft_consensus.cc:3037] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:51.716224 20670 raft_consensus.cc:504] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.716753 20670 leader_election.cc:288] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 election: Requested vote from peers 394ea4e8bbf54383b5e46f41d30ba49e (127.18.150.130:42641), e591116bfc9b4d9b9f53919715edab09 (127.18.150.129:35183)
I20260618 22:32:51.717185 20454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e45121b9bc949eba8f6fc309e64ce54" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394ea4e8bbf54383b5e46f41d30ba49e"
I20260618 22:32:51.717199 20319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e45121b9bc949eba8f6fc309e64ce54" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e591116bfc9b4d9b9f53919715edab09"
I20260618 22:32:51.717350 20319 raft_consensus.cc:3037] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:51.717363 20454 raft_consensus.cc:3037] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:51.719463 20319 raft_consensus.cc:2454] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 1.
I20260618 22:32:51.719655 20454 raft_consensus.cc:2454] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 1.
I20260618 22:32:51.719807 20525 leader_election.cc:302] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 739e0782e21440c8904811a075c91e0f, e591116bfc9b4d9b9f53919715edab09; no voters:
I20260618 22:32:51.720081 20670 raft_consensus.cc:2781] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:51.720441 20670 raft_consensus.cc:686] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 1 LEADER]: Becoming Leader. State: Replica: 739e0782e21440c8904811a075c91e0f, State: Running, Role: LEADER
I20260618 22:32:51.720856 20670 consensus_queue.cc:237] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.725419 20158 catalog_manager.cc:5495] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 739e0782e21440c8904811a075c91e0f (127.18.150.131). New cstate: current_term: 1 leader_uuid: "739e0782e21440c8904811a075c91e0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } health_report { overall_health: UNKNOWN } } }
I20260618 22:32:51.773970 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.769003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59910 (local address 127.18.150.131:38939)
0618 22:32:51.769211 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.769216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.769234 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:32:51.769478 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.769482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.769558 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.769673 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:51.770042 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.770721 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.772026 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.772436 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.772543 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.772929 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:51.773438 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:51.773553 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.773620 (+ 67us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.773695 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:32:51.781522 20454 raft_consensus.cc:1261] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e [term 1 FOLLOWER]: Refusing update from remote peer 739e0782e21440c8904811a075c91e0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:51.781520 20319 raft_consensus.cc:1261] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09 [term 1 FOLLOWER]: Refusing update from remote peer 739e0782e21440c8904811a075c91e0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:51.782390 20670 consensus_queue.cc:1035] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:51.782691 20672 consensus_queue.cc:1035] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:51.791716 20675 mvcc.cc:205] Tried to move back new op lower bound from 7298342796408627200 to 7298342796171960320. Current Snapshot: MvccSnapshot[applied={T|T < 7298342796408627200}]
I20260618 22:32:51.792349 20677 mvcc.cc:205] Tried to move back new op lower bound from 7298342796408627200 to 7298342796171960320. Current Snapshot: MvccSnapshot[applied={T|T < 7298342796408627200}]
I20260618 22:32:51.813657 20157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:32:51.813921 20157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:32:51.817601 20157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6: Sending DeleteTablet for 3 replicas of tablet 8e45121b9bc949eba8f6fc309e64ce54
I20260618 22:32:51.818789 20295 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e45121b9bc949eba8f6fc309e64ce54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:51 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:59448
I20260618 22:32:51.818782 20573 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e45121b9bc949eba8f6fc309e64ce54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:51 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:59898
I20260618 22:32:51.819537 20434 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e45121b9bc949eba8f6fc309e64ce54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:51 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:51908
I20260618 22:32:51.819921 20685 tablet_replica.cc:330] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f: stopping tablet replica
I20260618 22:32:51.820453 20685 raft_consensus.cc:2229] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:32:51.821300 20685 raft_consensus.cc:2258] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:51.822012 20157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:32:51.822696 20157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:32:51.824101 20685 ts_tablet_manager.cc:1905] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:51.830701 20573 tablet_service.cc:1467] Processing CreateTablet for tablet d4006ba2e8d24793a78d597e139984f9 (DEFAULT_TABLE table=test-table [id=5e54227bdd0a423cbca373e9be9c80a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:51.831212 20573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4006ba2e8d24793a78d597e139984f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:51.831252 20294 tablet_service.cc:1467] Processing CreateTablet for tablet d4006ba2e8d24793a78d597e139984f9 (DEFAULT_TABLE table=test-table [id=5e54227bdd0a423cbca373e9be9c80a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:51.831833 20294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4006ba2e8d24793a78d597e139984f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:51.835111 20433 tablet_service.cc:1467] Processing CreateTablet for tablet d4006ba2e8d24793a78d597e139984f9 (DEFAULT_TABLE table=test-table [id=5e54227bdd0a423cbca373e9be9c80a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:51.835683 20433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4006ba2e8d24793a78d597e139984f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:51.843611 20662 tablet_bootstrap.cc:492] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09: Bootstrap starting.
I20260618 22:32:51.843818 20664 tablet_bootstrap.cc:492] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f: Bootstrap starting.
I20260618 22:32:51.845237 20664 tablet_bootstrap.cc:654] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:51.845239 20662 tablet_bootstrap.cc:654] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:51.848747 20663 tablet_bootstrap.cc:492] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e: Bootstrap starting.
I20260618 22:32:51.850492 20663 tablet_bootstrap.cc:654] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:51.864828 20662 tablet_bootstrap.cc:492] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09: No bootstrap required, opened a new log
I20260618 22:32:51.865003 20662 ts_tablet_manager.cc:1397] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.000s
I20260618 22:32:51.865844 20663 tablet_bootstrap.cc:492] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e: No bootstrap required, opened a new log
I20260618 22:32:51.865859 20662 raft_consensus.cc:348] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } }
I20260618 22:32:51.866106 20662 raft_consensus.cc:374] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:51.866128 20664 tablet_bootstrap.cc:492] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f: No bootstrap required, opened a new log
I20260618 22:32:51.866209 20662 raft_consensus.cc:729] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e591116bfc9b4d9b9f53919715edab09, State: Initialized, Role: FOLLOWER
I20260618 22:32:51.866255 20664 ts_tablet_manager.cc:1397] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.002s
I20260618 22:32:51.866245 20663 ts_tablet_manager.cc:1397] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.001s
I20260618 22:32:51.866521 20662 consensus_queue.cc:260] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } }
I20260618 22:32:51.867058 20662 ts_tablet_manager.cc:1428] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:32:51.866950 20664 raft_consensus.cc:348] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } }
I20260618 22:32:51.867154 20664 raft_consensus.cc:374] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:51.867230 20664 raft_consensus.cc:729] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739e0782e21440c8904811a075c91e0f, State: Initialized, Role: FOLLOWER
I20260618 22:32:51.867489 20664 consensus_queue.cc:260] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } }
I20260618 22:32:51.869511 20663 raft_consensus.cc:348] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } }
I20260618 22:32:51.869875 20663 raft_consensus.cc:374] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:51.870033 20663 raft_consensus.cc:729] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394ea4e8bbf54383b5e46f41d30ba49e, State: Initialized, Role: FOLLOWER
I20260618 22:32:51.870571 20664 ts_tablet_manager.cc:1428] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260618 22:32:51.871279 20686 tablet_replica.cc:330] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e: stopping tablet replica
I20260618 22:32:51.872738 20686 raft_consensus.cc:2229] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:32:51.873216 20686 raft_consensus.cc:2258] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:51.874094 20663 consensus_queue.cc:260] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } }
I20260618 22:32:51.874799 20663 ts_tablet_manager.cc:1428] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260618 22:32:51.875231 20686 ts_tablet_manager.cc:1905] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:51.875624 20685 ts_tablet_manager.cc:1918] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:32:51.875833 20685 log.cc:1196] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/8e45121b9bc949eba8f6fc309e64ce54
I20260618 22:32:51.876451 20685 ts_tablet_manager.cc:1939] T 8e45121b9bc949eba8f6fc309e64ce54 P 739e0782e21440c8904811a075c91e0f: Deleting consensus metadata
I20260618 22:32:51.876762 20670 raft_consensus.cc:482] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:32:51.876899 20670 raft_consensus.cc:504] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } }
I20260618 22:32:51.877556 20670 leader_election.cc:288] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e591116bfc9b4d9b9f53919715edab09 (127.18.150.129:35183), 394ea4e8bbf54383b5e46f41d30ba49e (127.18.150.130:42641)
I20260618 22:32:51.878175 20319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4006ba2e8d24793a78d597e139984f9" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e591116bfc9b4d9b9f53919715edab09" is_pre_election: true
I20260618 22:32:51.878407 20319 raft_consensus.cc:2454] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 0.
I20260618 22:32:51.878957 20142 catalog_manager.cc:4841] TS 739e0782e21440c8904811a075c91e0f (127.18.150.131:38939): tablet 8e45121b9bc949eba8f6fc309e64ce54 (table test-table [id=b449ed3e89a84507b938598967f58352]) successfully deleted
I20260618 22:32:51.879078 20454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4006ba2e8d24793a78d597e139984f9" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" is_pre_election: true
I20260618 22:32:51.879292 20454 raft_consensus.cc:2454] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 0.
I20260618 22:32:51.879439 20525 leader_election.cc:302] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 739e0782e21440c8904811a075c91e0f, e591116bfc9b4d9b9f53919715edab09; no voters:
I20260618 22:32:51.879696 20670 raft_consensus.cc:2781] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:32:51.879808 20670 raft_consensus.cc:482] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:32:51.879895 20670 raft_consensus.cc:3037] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:51.883016 20670 raft_consensus.cc:504] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } }
I20260618 22:32:51.884994 20319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4006ba2e8d24793a78d597e139984f9" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e591116bfc9b4d9b9f53919715edab09"
I20260618 22:32:51.885221 20319 raft_consensus.cc:3037] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:51.886032 20670 leader_election.cc:288] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 election: Requested vote from peers e591116bfc9b4d9b9f53919715edab09 (127.18.150.129:35183), 394ea4e8bbf54383b5e46f41d30ba49e (127.18.150.130:42641)
I20260618 22:32:51.886135 20454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4006ba2e8d24793a78d597e139984f9" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394ea4e8bbf54383b5e46f41d30ba49e"
I20260618 22:32:51.886337 20454 raft_consensus.cc:3037] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:51.886636 20684 tablet_replica.cc:330] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09: stopping tablet replica
I20260618 22:32:51.887157 20684 raft_consensus.cc:2229] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:32:51.889015 20454 raft_consensus.cc:2454] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 1.
I20260618 22:32:51.889513 20526 leader_election.cc:302] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 394ea4e8bbf54383b5e46f41d30ba49e, 739e0782e21440c8904811a075c91e0f; no voters:
I20260618 22:32:51.889845 20670 raft_consensus.cc:2781] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:51.890023 20670 raft_consensus.cc:686] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 1 LEADER]: Becoming Leader. State: Replica: 739e0782e21440c8904811a075c91e0f, State: Running, Role: LEADER
I20260618 22:32:51.890313 20670 consensus_queue.cc:237] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } }
I20260618 22:32:51.890611 20686 ts_tablet_manager.cc:1918] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:32:51.890748 20686 log.cc:1196] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/8e45121b9bc949eba8f6fc309e64ce54
I20260618 22:32:51.891201 20686 ts_tablet_manager.cc:1939] T 8e45121b9bc949eba8f6fc309e64ce54 P 394ea4e8bbf54383b5e46f41d30ba49e: Deleting consensus metadata
I20260618 22:32:51.888865 20319 raft_consensus.cc:2454] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 1.
I20260618 22:32:51.892812 20144 catalog_manager.cc:4841] TS 394ea4e8bbf54383b5e46f41d30ba49e (127.18.150.130:42641): tablet 8e45121b9bc949eba8f6fc309e64ce54 (table test-table [id=b449ed3e89a84507b938598967f58352]) successfully deleted
I20260618 22:32:51.893090 20684 raft_consensus.cc:2258] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:51.895206 20684 ts_tablet_manager.cc:1905] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:51.895468 20158 catalog_manager.cc:5495] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 739e0782e21440c8904811a075c91e0f (127.18.150.131). New cstate: current_term: 1 leader_uuid: "739e0782e21440c8904811a075c91e0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } health_report { overall_health: HEALTHY } } }
I20260618 22:32:51.900413 20684 ts_tablet_manager.cc:1918] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:32:51.900535 20684 log.cc:1196] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/8e45121b9bc949eba8f6fc309e64ce54
I20260618 22:32:51.900902 20684 ts_tablet_manager.cc:1939] T 8e45121b9bc949eba8f6fc309e64ce54 P e591116bfc9b4d9b9f53919715edab09: Deleting consensus metadata
I20260618 22:32:51.902225 20143 catalog_manager.cc:4841] TS e591116bfc9b4d9b9f53919715edab09 (127.18.150.129:35183): tablet 8e45121b9bc949eba8f6fc309e64ce54 (table test-table [id=b449ed3e89a84507b938598967f58352]) successfully deleted
Jun 18 22:32:51 dist-test-slave-h7jz krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821969, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:51.945104 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.932257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.18.150.190:35947)
0618 22:32:51.932415 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:51.932422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:51.933109 (+ 687us) server_negotiation.cc:407] Connection header received
0618 22:32:51.933258 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:51.933262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:51.933331 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:51.933409 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:51.934554 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.935262 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.936445 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.936850 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.936951 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.940268 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:51.940272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:51.940275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:51.940302 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:51.942627 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.943231 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.943236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.943239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.943348 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:51.943673 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:51.943678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:51.943681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:51.943958 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:51.944087 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:51.944467 (+ 380us) server_negotiation.cc:299] Negotiation successful
0618 22:32:51.944797 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":110,"server-negotiator.queue_time_us":49}
I20260618 22:32:51.945111 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:51.932431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35947 (local address 127.0.0.1:49952)
0618 22:32:51.932996 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:51.933011 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:51.933151 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:51.933456 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:51.933460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:51.934171 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:51.934426 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.934435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.935379 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.935383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.936313 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:51.936324 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:51.937005 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:51.937012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:51.937141 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:51.937750 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:51.937771 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:51.940105 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:51.942738 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:51.942751 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:51.942759 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:51.943133 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:51.943438 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:51.943443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:51.943446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:51.943582 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:51.944072 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:51.944081 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:51.944240 (+ 159us) client_negotiation.cc:769] Sending connection context
0618 22:32:51.944374 (+ 134us) client_negotiation.cc:240] Negotiation successful
0618 22:32:51.944650 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":298,"threads_started":1}
I20260618 22:32:51.950548 20158 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:49952:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260618 22:32:51.951495 20158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:32:51.957852 20434 tablet_service.cc:1467] Processing CreateTablet for tablet 48e96715b1054e9babd051b7fbe741d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c7a5ebf032b4994b372a7feb3315263]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:32:51.958478 20434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e96715b1054e9babd051b7fbe741d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:51.958390 20573 tablet_service.cc:1467] Processing CreateTablet for tablet 48e96715b1054e9babd051b7fbe741d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c7a5ebf032b4994b372a7feb3315263]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:32:51.958873 20573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e96715b1054e9babd051b7fbe741d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:51.963063 20295 tablet_service.cc:1467] Processing CreateTablet for tablet 48e96715b1054e9babd051b7fbe741d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c7a5ebf032b4994b372a7feb3315263]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:32:51.963580 20295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e96715b1054e9babd051b7fbe741d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:51.964717 20664 tablet_bootstrap.cc:492] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: Bootstrap starting.
I20260618 22:32:51.966217 20664 tablet_bootstrap.cc:654] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:51.970335 20663 tablet_bootstrap.cc:492] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: Bootstrap starting.
I20260618 22:32:51.971868 20663 tablet_bootstrap.cc:654] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:51.973047 20664 tablet_bootstrap.cc:492] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: No bootstrap required, opened a new log
I20260618 22:32:51.973218 20664 ts_tablet_manager.cc:1397] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.003s
I20260618 22:32:51.974170 20662 tablet_bootstrap.cc:492] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: Bootstrap starting.
I20260618 22:32:51.974133 20664 raft_consensus.cc:348] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.974433 20664 raft_consensus.cc:374] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:51.974574 20664 raft_consensus.cc:729] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739e0782e21440c8904811a075c91e0f, State: Initialized, Role: FOLLOWER
I20260618 22:32:51.975087 20664 consensus_queue.cc:260] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.975544 20670 tablet_replica.cc:439] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } }
I20260618 22:32:51.975886 20662 tablet_bootstrap.cc:654] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:51.975968 20670 tablet_replica.cc:442] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:51.976490 20664 ts_tablet_manager.cc:1428] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260618 22:32:51.983294 20662 tablet_bootstrap.cc:492] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: No bootstrap required, opened a new log
I20260618 22:32:51.983470 20662 ts_tablet_manager.cc:1397] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260618 22:32:51.984157 20662 raft_consensus.cc:348] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.984333 20663 tablet_bootstrap.cc:492] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: No bootstrap required, opened a new log
I20260618 22:32:51.984344 20662 raft_consensus.cc:374] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:51.984418 20662 raft_consensus.cc:729] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e591116bfc9b4d9b9f53919715edab09, State: Initialized, Role: FOLLOWER
I20260618 22:32:51.984486 20663 ts_tablet_manager.cc:1397] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260618 22:32:51.984642 20662 consensus_queue.cc:260] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.985016 20668 tablet_replica.cc:439] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } }
I20260618 22:32:51.985157 20663 raft_consensus.cc:348] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.985361 20668 tablet_replica.cc:442] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:51.985381 20663 raft_consensus.cc:374] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:51.985443 20663 raft_consensus.cc:729] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394ea4e8bbf54383b5e46f41d30ba49e, State: Initialized, Role: FOLLOWER
I20260618 22:32:51.985647 20663 consensus_queue.cc:260] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:51.986220 20663 ts_tablet_manager.cc:1428] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:32:51.986097 20669 tablet_replica.cc:439] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } }
I20260618 22:32:51.986444 20669 tablet_replica.cc:442] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:51.985862 20662 ts_tablet_manager.cc:1428] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260618 22:32:52.042798 20644 tablet.cc:2369] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:32:52.061280 20670 raft_consensus.cc:482] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:32:52.061538 20670 raft_consensus.cc:504] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:52.062430 20670 leader_election.cc:288] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 394ea4e8bbf54383b5e46f41d30ba49e (127.18.150.130:42641), e591116bfc9b4d9b9f53919715edab09 (127.18.150.129:35183)
W20260618 22:32:52.062568 20366 tablet.cc:2369] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:32:52.062983 20454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e96715b1054e9babd051b7fbe741d7" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" is_pre_election: true
I20260618 22:32:52.063263 20454 raft_consensus.cc:2454] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 0.
I20260618 22:32:52.063202 20319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e96715b1054e9babd051b7fbe741d7" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e591116bfc9b4d9b9f53919715edab09" is_pre_election: true
I20260618 22:32:52.063481 20319 raft_consensus.cc:2454] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 0.
I20260618 22:32:52.063723 20526 leader_election.cc:302] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 394ea4e8bbf54383b5e46f41d30ba49e, 739e0782e21440c8904811a075c91e0f; no voters:
I20260618 22:32:52.063997 20670 raft_consensus.cc:2781] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:32:52.064126 20670 raft_consensus.cc:482] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:32:52.064234 20670 raft_consensus.cc:3037] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:52.066362 20670 raft_consensus.cc:504] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:52.066955 20670 leader_election.cc:288] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 election: Requested vote from peers 394ea4e8bbf54383b5e46f41d30ba49e (127.18.150.130:42641), e591116bfc9b4d9b9f53919715edab09 (127.18.150.129:35183)
I20260618 22:32:52.067344 20454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e96715b1054e9babd051b7fbe741d7" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394ea4e8bbf54383b5e46f41d30ba49e"
I20260618 22:32:52.067489 20319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e96715b1054e9babd051b7fbe741d7" candidate_uuid: "739e0782e21440c8904811a075c91e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e591116bfc9b4d9b9f53919715edab09"
I20260618 22:32:52.067582 20454 raft_consensus.cc:3037] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:52.067716 20319 raft_consensus.cc:3037] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:52.070396 20454 raft_consensus.cc:2454] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 1.
I20260618 22:32:52.070396 20319 raft_consensus.cc:2454] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 739e0782e21440c8904811a075c91e0f in term 1.
I20260618 22:32:52.070802 20526 leader_election.cc:302] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 394ea4e8bbf54383b5e46f41d30ba49e, 739e0782e21440c8904811a075c91e0f; no voters:
I20260618 22:32:52.071089 20670 raft_consensus.cc:2781] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:52.071280 20670 raft_consensus.cc:686] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [term 1 LEADER]: Becoming Leader. State: Replica: 739e0782e21440c8904811a075c91e0f, State: Running, Role: LEADER
I20260618 22:32:52.071653 20670 consensus_queue.cc:237] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } }
I20260618 22:32:52.073643 20672 tablet_replica.cc:439] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: TxnStatusTablet state changed. Reason: New leader 739e0782e21440c8904811a075c91e0f. Latest consensus state: current_term: 1 leader_uuid: "739e0782e21440c8904811a075c91e0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } }
I20260618 22:32:52.073987 20672 tablet_replica.cc:442] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:32:52.075210 20158 catalog_manager.cc:5495] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 739e0782e21440c8904811a075c91e0f (127.18.150.131). New cstate: current_term: 1 leader_uuid: "739e0782e21440c8904811a075c91e0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } health_report { overall_health: UNKNOWN } } }
W20260618 22:32:52.078778 20505 tablet.cc:2369] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:32:52.086133 20701 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260618 22:32:52.086444 20701 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260618 22:32:52.102773 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:52.096548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:38939 (local address 127.0.0.1:59926)
0618 22:32:52.096768 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:52.096784 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:52.096918 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:52.097440 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:52.097445 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:52.097458 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:32:52.097639 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:52.097649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.099141 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.099147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:52.100695 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:52.100710 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.101652 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.101656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:52.101799 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:52.101854 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:32:52.102294 (+ 440us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:32:52.102330 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:32:52.102408 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:32:52.102538 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260618 22:32:52.102813 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:52.096516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59926 (local address 127.18.150.131:38939)
0618 22:32:52.096696 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:52.096702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:52.096846 (+ 144us) server_negotiation.cc:407] Connection header received
0618 22:32:52.097078 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:52.097083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:52.097172 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:52.097255 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:52.097794 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.098976 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.100854 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.101493 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.101614 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:52.102036 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:52.102175 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:52.102278 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:52.102446 (+ 168us) server_negotiation.cc:299] Negotiation successful
0618 22:32:52.102530 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:32:52.106303 20319 raft_consensus.cc:1261] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09 [term 1 FOLLOWER]: Refusing update from remote peer 739e0782e21440c8904811a075c91e0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:52.106312 20454 raft_consensus.cc:1261] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e [term 1 FOLLOWER]: Refusing update from remote peer 739e0782e21440c8904811a075c91e0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:52.106846 20672 consensus_queue.cc:1035] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:52.107117 20670 consensus_queue.cc:1035] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:52.111047 20669 tablet_replica.cc:439] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: TxnStatusTablet state changed. Reason: New leader 739e0782e21440c8904811a075c91e0f. Latest consensus state: current_term: 1 leader_uuid: "739e0782e21440c8904811a075c91e0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } }
I20260618 22:32:52.111336 20669 tablet_replica.cc:442] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:52.111992 20670 tablet_replica.cc:439] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "739e0782e21440c8904811a075c91e0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } }
I20260618 22:32:52.112268 20670 tablet_replica.cc:442] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:32:52.112736 20670 tablet_replica.cc:439] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "739e0782e21440c8904811a075c91e0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } }
I20260618 22:32:52.112977 20670 tablet_replica.cc:442] T 48e96715b1054e9babd051b7fbe741d7 P 739e0782e21440c8904811a075c91e0f: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:32:52.113013 20668 tablet_replica.cc:439] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: TxnStatusTablet state changed. Reason: New leader 739e0782e21440c8904811a075c91e0f. Latest consensus state: current_term: 1 leader_uuid: "739e0782e21440c8904811a075c91e0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } }
I20260618 22:32:52.113299 20668 tablet_replica.cc:442] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:52.117933 20676 mvcc.cc:205] Tried to move back new op lower bound from 7298342797742104576 to 7298342797609291776. Current Snapshot: MvccSnapshot[applied={T|T < 7298342797742104576}]
I20260618 22:32:52.119215 20669 tablet_replica.cc:439] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "739e0782e21440c8904811a075c91e0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } }
I20260618 22:32:52.119445 20669 tablet_replica.cc:442] T 48e96715b1054e9babd051b7fbe741d7 P 394ea4e8bbf54383b5e46f41d30ba49e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:52.120631 20668 tablet_replica.cc:439] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "739e0782e21440c8904811a075c91e0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 } } peers { permanent_uuid: "739e0782e21440c8904811a075c91e0f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38939 } } peers { permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 } } }
I20260618 22:32:52.120867 20668 tablet_replica.cc:442] T 48e96715b1054e9babd051b7fbe741d7 P e591116bfc9b4d9b9f53919715edab09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:52.136260 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:52.130004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:55909 (local address 127.18.150.131:38939)
0618 22:32:52.130178 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:52.130184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:52.130733 (+ 549us) server_negotiation.cc:407] Connection header received
0618 22:32:52.130879 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:52.130885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:52.130975 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:52.131073 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:52.131649 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.132739 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.134250 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.134891 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.135046 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:52.135532 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:52.135696 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:52.135785 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:52.135897 (+ 112us) server_negotiation.cc:299] Negotiation successful
0618 22:32:52.135981 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":45}
I20260618 22:32:52.136298 20707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:52.129887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:38939 (local address 127.18.150.131:55909)
0618 22:32:52.130633 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:52.130649 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:52.130762 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:52.131114 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:52.131117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:52.131127 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:32:52.131447 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:52.131455 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.132889 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.132893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:52.134124 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:52.134136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.135133 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.135138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:52.135289 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:52.135357 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:32:52.135792 (+ 435us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:32:52.135800 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:32:52.135859 (+ 59us) client_negotiation.cc:240] Negotiation successful
0618 22:32:52.135962 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":444,"threads_started":1}
I20260618 22:32:52.146144 20454 raft_consensus.cc:1261] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e [term 1 FOLLOWER]: Refusing update from remote peer 739e0782e21440c8904811a075c91e0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:52.146685 20670 consensus_queue.cc:1035] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "394ea4e8bbf54383b5e46f41d30ba49e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:52.158638 20319 raft_consensus.cc:1261] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09 [term 1 FOLLOWER]: Refusing update from remote peer 739e0782e21440c8904811a075c91e0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:52.159337 20672 consensus_queue.cc:1035] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e591116bfc9b4d9b9f53919715edab09" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:52.219786 20157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:32:52.220037 20157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:32:52.223209 20157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab21c91869ac4c83bbbc7329724442c6: Sending DeleteTablet for 3 replicas of tablet d4006ba2e8d24793a78d597e139984f9
I20260618 22:32:52.224088 20434 tablet_service.cc:1514] Processing DeleteTablet for tablet d4006ba2e8d24793a78d597e139984f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:52 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:51908
I20260618 22:32:52.224243 20572 tablet_service.cc:1514] Processing DeleteTablet for tablet d4006ba2e8d24793a78d597e139984f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:52 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:59898
I20260618 22:32:52.224280 20295 tablet_service.cc:1514] Processing DeleteTablet for tablet d4006ba2e8d24793a78d597e139984f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:52 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:59448
I20260618 22:32:52.224490 20685 tablet_replica.cc:330] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f: stopping tablet replica
I20260618 22:32:52.224555 20684 tablet_replica.cc:330] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09: stopping tablet replica
I20260618 22:32:52.224578 20686 tablet_replica.cc:330] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e: stopping tablet replica
I20260618 22:32:52.224677 20685 raft_consensus.cc:2229] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:32:52.224749 20684 raft_consensus.cc:2229] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:32:52.224753 20686 raft_consensus.cc:2229] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:32:52.224941 20684 raft_consensus.cc:2258] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:52.224946 20686 raft_consensus.cc:2258] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:52.224982 20685 raft_consensus.cc:2258] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:52.226289 20685 ts_tablet_manager.cc:1905] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:52.226280 20686 ts_tablet_manager.cc:1905] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:52.230531 20684 ts_tablet_manager.cc:1905] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:52.233397 20686 ts_tablet_manager.cc:1918] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:32:52.233647 20686 log.cc:1196] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/d4006ba2e8d24793a78d597e139984f9
I20260618 22:32:52.234186 20686 ts_tablet_manager.cc:1939] T d4006ba2e8d24793a78d597e139984f9 P 394ea4e8bbf54383b5e46f41d30ba49e: Deleting consensus metadata
I20260618 22:32:52.235388 20144 catalog_manager.cc:4841] TS 394ea4e8bbf54383b5e46f41d30ba49e (127.18.150.130:42641): tablet d4006ba2e8d24793a78d597e139984f9 (table test-table [id=5e54227bdd0a423cbca373e9be9c80a4]) successfully deleted
I20260618 22:32:52.238731 20685 ts_tablet_manager.cc:1918] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:32:52.238888 20685 log.cc:1196] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/d4006ba2e8d24793a78d597e139984f9
I20260618 22:32:52.239255 20685 ts_tablet_manager.cc:1939] T d4006ba2e8d24793a78d597e139984f9 P 739e0782e21440c8904811a075c91e0f: Deleting consensus metadata
I20260618 22:32:52.240260 20142 catalog_manager.cc:4841] TS 739e0782e21440c8904811a075c91e0f (127.18.150.131:38939): tablet d4006ba2e8d24793a78d597e139984f9 (table test-table [id=5e54227bdd0a423cbca373e9be9c80a4]) successfully deleted
I20260618 22:32:52.241528 20684 ts_tablet_manager.cc:1918] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:32:52.241726 20684 log.cc:1196] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/d4006ba2e8d24793a78d597e139984f9
I20260618 22:32:52.242146 20684 ts_tablet_manager.cc:1939] T d4006ba2e8d24793a78d597e139984f9 P e591116bfc9b4d9b9f53919715edab09: Deleting consensus metadata
I20260618 22:32:52.243125 20143 catalog_manager.cc:4841] TS e591116bfc9b4d9b9f53919715edab09 (127.18.150.129:35183): tablet d4006ba2e8d24793a78d597e139984f9 (table test-table [id=5e54227bdd0a423cbca373e9be9c80a4]) successfully deleted
Jun 18 22:32:52 dist-test-slave-h7jz krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821971, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:32:52.257552 20714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:52.235644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59462 (local address 127.18.150.129:35183)
0618 22:32:52.236119 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:52.236125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:52.243477 (+ 7352us) server_negotiation.cc:407] Connection header received
0618 22:32:52.243650 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:52.243656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:52.243733 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:52.243828 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:52.245229 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.245932 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.247211 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.247650 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.247756 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:52.251720 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:52.251736 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:52.251753 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:52.251790 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:52.254751 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:52.255511 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:52.255523 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:52.255526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:52.255701 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:52.256091 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:52.256094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:52.256096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:52.256427 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:52.256716 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:52.257000 (+ 284us) server_negotiation.cc:299] Negotiation successful
0618 22:32:52.257217 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":236,"threads_started":1}
W20260618 22:32:52.258294 20360 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59462
I20260618 22:32:52.275044 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:52.261224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49968 (local address 127.18.150.190:35947)
0618 22:32:52.261418 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:52.261423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:52.261643 (+ 220us) server_negotiation.cc:407] Connection header received
0618 22:32:52.261851 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:52.261855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:52.261921 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:52.262062 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:52.263630 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.264360 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.266233 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.266713 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.266810 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:52.269724 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:52.269732 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:52.269738 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:52.269784 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:52.272361 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:52.273122 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:52.273126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:52.273128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:52.273242 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:52.273592 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:52.273595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:52.273597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:52.273864 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:52.274139 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:52.274434 (+ 295us) server_negotiation.cc:299] Negotiation successful
0618 22:32:52.274671 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260618 22:32:52.275579 20157 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49968
I20260618 22:32:52.277254 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20229
I20260618 22:32:52.294209 20714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:52.278053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59474 (local address 127.18.150.129:35183)
0618 22:32:52.278229 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:52.278234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:52.282348 (+ 4114us) server_negotiation.cc:407] Connection header received
0618 22:32:52.282509 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:52.282513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:52.282575 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:52.282651 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:52.283940 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.284627 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.286237 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.286673 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.286762 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:52.289317 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:52.289322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:52.289325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:52.289355 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:52.291698 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:52.292490 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:52.292496 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:52.292507 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:52.292629 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:52.292959 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:52.292962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:52.292964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:52.293199 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:52.293422 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:52.293704 (+ 282us) server_negotiation.cc:299] Negotiation successful
0618 22:32:52.293892 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260618 22:32:52.294751 20360 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474
W20260618 22:32:52.295215 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20229: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:52.295360 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 20229
I20260618 22:32:52.308832 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20369
Jun 18 22:32:52 dist-test-slave-h7jz krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821971, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:32:52.326442 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:52.309505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51910 (local address 127.18.150.130:42641)
0618 22:32:52.311818 (+ 2313us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:52.311823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:52.311838 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:32:52.311897 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:52.311903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:52.311984 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:52.312071 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:52.313598 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.314334 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.316088 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.316587 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.316695 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:52.321074 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:52.321089 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:52.321104 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:52.321137 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:52.323865 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:52.324467 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:52.324474 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:52.324476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:52.324607 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:52.324949 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:52.324953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:52.324955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:52.325279 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:52.325543 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:52.325865 (+ 322us) server_negotiation.cc:299] Negotiation successful
0618 22:32:52.326142 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2173,"thread_start_us":2067,"threads_started":1}
W20260618 22:32:52.327095 20499 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51910
W20260618 22:32:52.327715 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20369: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:52.327865 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 20369
I20260618 22:32:52.341074 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20508
Jun 18 22:32:52 dist-test-slave-h7jz krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821971, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:32:52.357731 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:52.341848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59940 (local address 127.18.150.131:38939)
0618 22:32:52.342055 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:52.342061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:52.342226 (+ 165us) server_negotiation.cc:407] Connection header received
0618 22:32:52.342392 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:52.342396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:52.342465 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:52.342574 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:52.344071 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.344842 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.346633 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:52.347073 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:52.347165 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:52.351948 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:52.351962 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:52.351977 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:52.352017 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:52.354919 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:52.355674 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:52.355686 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:52.355688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:52.355834 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:52.356215 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:52.356219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:52.356222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:52.356594 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:52.356819 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:52.357215 (+ 396us) server_negotiation.cc:299] Negotiation successful
0618 22:32:52.357382 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260618 22:32:52.358431 20638 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59940
W20260618 22:32:52.359025 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20508: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:32:52.359177 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 20508
I20260618 22:32:52.375980 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20126
I20260618 22:32:52.377036 20198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:32:52.487989 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 20126
2026-06-18T22:32:52Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4087 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:32:52 dist-test-slave-h7jz krb5kdc[20723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:32:52 dist-test-slave-h7jz krb5kdc[20723](info): set up 2 sockets
Jun 18 22:32:52 dist-test-slave-h7jz krb5kdc[20723](info): commencing operation
krb5kdc: starting...
W20260618 22:32:53.477751 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:32:53 dist-test-slave-h7jz krb5kdc[20723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:32:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:32:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:32:53.647187 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35743
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:33161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:35743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:53.956187 20736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:53.956483 20736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:53.956632 20736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:53.963718 20736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:32:53.963832 20736 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260618 22:32:53.963894 20736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:53.963975 20736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:53.964035 20736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:32:53.964113 20736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:32:53.973788 20736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:33161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:35743
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:53.975026 20736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:53.976260 20736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:53.987323 20742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:53.987419 20744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:53.987602 20741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:53.988000 20736 server_base.cc:1034] running on GCE node
I20260618 22:32:53.988670 20736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:53.989924 20736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:53.991159 20736 hybrid_clock.cc:648] HybridClock initialized: now 1781821973991097 us; error 39 us; skew 500 ppm
I20260618 22:32:53.991464 20736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:53 dist-test-slave-h7jz krb5kdc[20723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821973, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:53.994297 20736 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:32:53.995604 20736 webserver.cc:460] Webserver started at http://127.18.150.190:33517/ using document root <none> and password file <none>
I20260618 22:32:53.996336 20736 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:53.996433 20736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:53.996707 20736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:53.999529 20736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "ffae639e22b24681b9074ef9d509f752"
format_stamp: "Formatted at 2026-06-18 22:32:53 on dist-test-slave-h7jz"
server_key: "d8c767d2f28a1e1e42d385e42b0eec06"
server_key_iv: "17dabbc9f34f6dd30f3ab6dcf88d7d06"
server_key_version: "encryptionkey@0"
I20260618 22:32:54.000238 20736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "ffae639e22b24681b9074ef9d509f752"
format_stamp: "Formatted at 2026-06-18 22:32:53 on dist-test-slave-h7jz"
server_key: "d8c767d2f28a1e1e42d385e42b0eec06"
server_key_iv: "17dabbc9f34f6dd30f3ab6dcf88d7d06"
server_key_version: "encryptionkey@0"
I20260618 22:32:54.004949 20736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260618 22:32:54.008153 20751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:54.009413 20736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:32:54.009562 20736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "ffae639e22b24681b9074ef9d509f752"
format_stamp: "Formatted at 2026-06-18 22:32:53 on dist-test-slave-h7jz"
server_key: "d8c767d2f28a1e1e42d385e42b0eec06"
server_key_iv: "17dabbc9f34f6dd30f3ab6dcf88d7d06"
server_key_version: "encryptionkey@0"
I20260618 22:32:54.009745 20736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:54.029331 20736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:54.032960 20736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:54.033205 20736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:54.051719 20736 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:35743
I20260618 22:32:54.051738 20812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:35743 every 8 connection(s)
I20260618 22:32:54.053063 20736 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:32:54.056829 20813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:54.062484 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 20736
I20260618 22:32:54.062661 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:32:54.063242 19034 external_mini_cluster.cc:1428] Setting key f2ed4df8d8a0343468f9afce0124c62c
I20260618 22:32:54.067098 20813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752: Bootstrap starting.
I20260618 22:32:54.070770 20813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:54.071851 20813 log.cc:824] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:54.074729 20813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752: No bootstrap required, opened a new log
Jun 18 22:32:54 dist-test-slave-h7jz krb5kdc[20723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:54.082719 20813 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae639e22b24681b9074ef9d509f752" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35743 } }
I20260618 22:32:54.083073 20813 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:54.083174 20813 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae639e22b24681b9074ef9d509f752, State: Initialized, Role: FOLLOWER
I20260618 22:32:54.083868 20813 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae639e22b24681b9074ef9d509f752" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35743 } }
I20260618 22:32:54.084146 20813 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:32:54.084285 20813 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:32:54.084455 20813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:54.085336 20816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:54.065122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34506 (local address 127.18.150.190:35743)
0618 22:32:54.065829 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:54.065840 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:54.065879 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:32:54.067472 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:54.067494 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:54.067885 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:54.068285 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:54.070002 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:54.071358 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:54.072967 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:54.073639 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:54.073772 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:54.078115 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:54.078123 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:54.078137 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:54.078179 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:54.081089 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:54.081723 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:54.081729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:54.081736 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:54.081900 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:54.082274 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:54.082279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:54.082281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:54.082785 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:54.083036 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:54.083493 (+ 457us) server_negotiation.cc:299] Negotiation successful
0618 22:32:54.083915 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":331,"threads_started":1}
I20260618 22:32:54.087073 20813 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae639e22b24681b9074ef9d509f752" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35743 } }
I20260618 22:32:54.087544 20813 leader_election.cc:302] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffae639e22b24681b9074ef9d509f752; no voters:
I20260618 22:32:54.088088 20813 leader_election.cc:288] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:32:54.088195 20818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:54.088801 20818 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [term 1 LEADER]: Becoming Leader. State: Replica: ffae639e22b24681b9074ef9d509f752, State: Running, Role: LEADER
I20260618 22:32:54.089284 20818 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae639e22b24681b9074ef9d509f752" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35743 } }
I20260618 22:32:54.090005 20813 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:32:54.091832 20820 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffae639e22b24681b9074ef9d509f752. Latest consensus state: current_term: 1 leader_uuid: "ffae639e22b24681b9074ef9d509f752" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae639e22b24681b9074ef9d509f752" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35743 } } }
I20260618 22:32:54.092247 20820 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:54.092146 20819 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffae639e22b24681b9074ef9d509f752" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae639e22b24681b9074ef9d509f752" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35743 } } }
I20260618 22:32:54.092382 20819 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752 [sys.catalog]: This master's current role is: LEADER
I20260618 22:32:54.094568 20827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:32:54.099577 20827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:32:54.108566 20827 catalog_manager.cc:1347] Generated new cluster ID: 74f74660dab9412486a9859f4b3080ab
I20260618 22:32:54.108680 20827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:32:54.116634 20827 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:32:54.117909 20827 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:32:54.126955 20827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752: Generated new TSK 0
I20260618 22:32:54.127760 20827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:32:54.176678 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35743
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:33161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:54.498558 20839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:54.498832 20839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:54.498971 20839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:54.506283 20839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:54.506420 20839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:54.506553 20839 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260618 22:32:54.506613 20839 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:32:54.506714 20839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:32:54.516705 20839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:33161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.150.190:35743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:54.518045 20839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:54.519383 20839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:54.532570 20844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:54.532593 20845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:54.532547 20847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:54.533438 20839 server_base.cc:1034] running on GCE node
I20260618 22:32:54.534174 20839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:54.535173 20839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:54.536418 20839 hybrid_clock.cc:648] HybridClock initialized: now 1781821974536346 us; error 57 us; skew 500 ppm
I20260618 22:32:54.536799 20839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:54 dist-test-slave-h7jz krb5kdc[20723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821974, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:54.540092 20839 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:32:54.541702 20839 webserver.cc:460] Webserver started at http://127.18.150.129:41671/ using document root <none> and password file <none>
I20260618 22:32:54.542467 20839 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:54.542587 20839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:54.542896 20839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:54.546032 20839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "b5b4b5b6f36641c39c5a020413c59c67"
format_stamp: "Formatted at 2026-06-18 22:32:54 on dist-test-slave-h7jz"
server_key: "ad24995fcd9566d3fd9b4750370778c9"
server_key_iv: "8666433b96f7a648f15f0c4a868c4363"
server_key_version: "encryptionkey@0"
I20260618 22:32:54.546869 20839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "b5b4b5b6f36641c39c5a020413c59c67"
format_stamp: "Formatted at 2026-06-18 22:32:54 on dist-test-slave-h7jz"
server_key: "ad24995fcd9566d3fd9b4750370778c9"
server_key_iv: "8666433b96f7a648f15f0c4a868c4363"
server_key_version: "encryptionkey@0"
I20260618 22:32:54.552300 20839 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:32:54.556092 20854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:54.557492 20839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:32:54.557651 20839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "b5b4b5b6f36641c39c5a020413c59c67"
format_stamp: "Formatted at 2026-06-18 22:32:54 on dist-test-slave-h7jz"
server_key: "ad24995fcd9566d3fd9b4750370778c9"
server_key_iv: "8666433b96f7a648f15f0c4a868c4363"
server_key_version: "encryptionkey@0"
I20260618 22:32:54.557842 20839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:54.581264 20839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:54.585850 20839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:54.586184 20839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:54.589098 20839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:54.589226 20839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:54.589401 20839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:54.589529 20839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:32:54 dist-test-slave-h7jz krb5kdc[20723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821974, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:54.615207 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:54.590226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35743 (local address 127.18.150.129:55219)
0618 22:32:54.591684 (+ 1458us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:54.591742 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:54.594327 (+ 2585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:54.595216 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:54.595228 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:54.596140 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:54.597191 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:54.597222 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:54.598727 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:54.598736 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:54.601272 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:54.601292 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:54.602362 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:54.602371 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:54.602587 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:54.603427 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:54.603472 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:54.606959 (+ 3487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:54.610405 (+ 3446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:54.610414 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:54.610436 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:54.610938 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:54.611376 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:54.611381 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:54.611385 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:54.611653 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:54.612240 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:54.612248 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:54.612662 (+ 414us) client_negotiation.cc:769] Sending connection context
0618 22:32:54.612941 (+ 279us) client_negotiation.cc:240] Negotiation successful
0618 22:32:54.613540 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"thread_start_us":388,"threads_started":1}
I20260618 22:32:54.615782 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:54.589513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:55219 (local address 127.18.150.190:35743)
0618 22:32:54.591569 (+ 2056us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:54.591576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:54.591903 (+ 327us) server_negotiation.cc:407] Connection header received
0618 22:32:54.594676 (+ 2773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:54.594682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:54.594766 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:54.594875 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:54.597428 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:54.598526 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:54.601499 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:54.602107 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:54.602220 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:54.607220 (+ 5000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:54.607227 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:54.607232 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:54.607276 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:54.610268 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:54.611122 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:54.611128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:54.611132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:54.611239 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:54.611757 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:54.611762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:54.611766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:54.612110 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:54.612239 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:54.615223 (+ 2984us) server_negotiation.cc:299] Negotiation successful
0618 22:32:54.615441 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1931,"thread_start_us":1836,"threads_started":1}
I20260618 22:32:54.626367 20839 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:41121
I20260618 22:32:54.626461 20974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:41121 every 8 connection(s)
I20260618 22:32:54.627568 20839 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:32:54.629962 20975 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35743
I20260618 22:32:54.630179 20975 heartbeater.cc:464] Registering TS with master...
I20260618 22:32:54.630908 20975 heartbeater.cc:511] Master 127.18.150.190:35743 requested a full tablet report, sending...
I20260618 22:32:54.632889 20768 ts_manager.cc:194] Registered new tserver with Master: b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121)
I20260618 22:32:54.634671 20768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:55219
I20260618 22:32:54.635214 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:54.622443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:48055 (local address 127.18.150.190:35743)
0618 22:32:54.622612 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:54.622617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:54.623279 (+ 662us) server_negotiation.cc:407] Connection header received
0618 22:32:54.623543 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:54.623548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:54.623626 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:54.623693 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:54.624900 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:54.625587 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:54.627418 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:54.627876 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:54.627996 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:54.630159 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:54.630165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:54.630170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:54.630208 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:54.632843 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:54.633515 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:54.633519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:54.633521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:54.633650 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:54.634014 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:54.634019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:54.634023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:54.634257 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:54.634359 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:54.634781 (+ 422us) server_negotiation.cc:299] Negotiation successful
0618 22:32:54.634980 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:32:54.635720 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 20839
I20260618 22:32:54.635905 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:32:54.636493 19034 external_mini_cluster.cc:1428] Setting key 870eb375e7bf4cf9d7b16d7a1d2d52e3
I20260618 22:32:54.637046 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:54.622499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35743 (local address 127.18.150.129:48055)
0618 22:32:54.623176 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:54.623196 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:54.623387 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:54.623865 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:54.623868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:54.624348 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:54.624763 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:54.624775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:54.625712 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:54.625718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:54.627299 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:54.627311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:54.628081 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:54.628091 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:54.628297 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:54.629077 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:54.629097 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:54.630003 (+ 906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:54.632961 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:54.632966 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:54.632970 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:54.633419 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:54.633748 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:54.633752 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:54.633756 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:54.633898 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:54.634353 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:54.634358 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:54.634555 (+ 197us) client_negotiation.cc:769] Sending connection context
0618 22:32:54.634702 (+ 147us) client_negotiation.cc:240] Negotiation successful
0618 22:32:54.634966 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":361,"threads_started":1}
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:32:54.672937 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35743
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:33161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:54.983601 20979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:54.983954 20979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:54.984103 20979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:54.991328 20979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:54.991449 20979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:54.991575 20979 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260618 22:32:54.991638 20979 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:32:54.991739 20979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:32:55.001562 20979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:33161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.150.190:35743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:55.002866 20979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:55.004164 20979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:55.017042 20984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:55.017014 20987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:55.016965 20985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:55.017751 20979 server_base.cc:1034] running on GCE node
I20260618 22:32:55.018468 20979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:55.019372 20979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:55.020576 20979 hybrid_clock.cc:648] HybridClock initialized: now 1781821975020515 us; error 49 us; skew 500 ppm
I20260618 22:32:55.020874 20979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:55 dist-test-slave-h7jz krb5kdc[20723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821975, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:55.023823 20979 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:32:55.025195 20979 webserver.cc:460] Webserver started at http://127.18.150.130:35011/ using document root <none> and password file <none>
I20260618 22:32:55.025849 20979 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:55.025982 20979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:55.026264 20979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:55.029104 20979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "76d5a92bac054a028eaf283943f8f500"
format_stamp: "Formatted at 2026-06-18 22:32:55 on dist-test-slave-h7jz"
server_key: "96ffcb0bcd33f9537266b41747ed7977"
server_key_iv: "a7480e2f61516fbd0e99a19dc7242774"
server_key_version: "encryptionkey@0"
I20260618 22:32:55.029824 20979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "76d5a92bac054a028eaf283943f8f500"
format_stamp: "Formatted at 2026-06-18 22:32:55 on dist-test-slave-h7jz"
server_key: "96ffcb0bcd33f9537266b41747ed7977"
server_key_iv: "a7480e2f61516fbd0e99a19dc7242774"
server_key_version: "encryptionkey@0"
I20260618 22:32:55.034796 20979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:32:55.038271 20994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:55.039641 20979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:32:55.039793 20979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "76d5a92bac054a028eaf283943f8f500"
format_stamp: "Formatted at 2026-06-18 22:32:55 on dist-test-slave-h7jz"
server_key: "96ffcb0bcd33f9537266b41747ed7977"
server_key_iv: "a7480e2f61516fbd0e99a19dc7242774"
server_key_version: "encryptionkey@0"
I20260618 22:32:55.039966 20979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:55.052553 20979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:55.056416 20979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:55.056658 20979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:55.059746 20979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:55.059859 20979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:55.059970 20979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:55.060046 20979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:32:55 dist-test-slave-h7jz krb5kdc[20723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821975, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:55.083642 21003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.060504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35743 (local address 127.18.150.130:52655)
0618 22:32:55.061405 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:55.061466 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:55.063992 (+ 2526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:55.065032 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:55.065047 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:55.065834 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:55.066649 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.066674 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.068246 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.068252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.070353 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.070365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.071499 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.071507 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.071776 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.072531 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:55.072564 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:55.075572 (+ 3008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:55.078967 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:55.078977 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:55.078994 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:55.079570 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:55.079989 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:55.079993 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:55.079997 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:55.080163 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:55.080798 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:55.080807 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:55.081209 (+ 402us) client_negotiation.cc:769] Sending connection context
0618 22:32:55.081548 (+ 339us) client_negotiation.cc:240] Negotiation successful
0618 22:32:55.082099 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":347,"threads_started":1}
I20260618 22:32:55.084712 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.059785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:52655 (local address 127.18.150.190:35743)
0618 22:32:55.059959 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:55.059965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:55.061579 (+ 1614us) server_negotiation.cc:407] Connection header received
0618 22:32:55.064329 (+ 2750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:55.064335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:55.064444 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:55.064524 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:55.066833 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.067957 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.070514 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.071088 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.071209 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.075832 (+ 4623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:55.075838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:55.075843 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:55.075884 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:55.078838 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.079704 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.079709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.079713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.079878 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.080263 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.080269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.080272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.080660 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:55.080807 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:55.084195 (+ 3388us) server_negotiation.cc:299] Negotiation successful
0618 22:32:55.084436 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:32:55.103317 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.089518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35743 (local address 127.18.150.130:51611)
0618 22:32:55.090128 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:55.090152 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:55.090297 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:55.090629 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:55.090634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:55.091186 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:55.091550 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.091562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.092554 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.092559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.094204 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.094221 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.095149 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.095155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.095314 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.095939 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:55.095955 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:55.096704 (+ 749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:55.100245 (+ 3541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:55.100250 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:55.100258 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:55.100816 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:55.101232 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:55.101240 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:55.101243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:55.101400 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:55.102156 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:55.102159 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:55.102376 (+ 217us) client_negotiation.cc:769] Sending connection context
0618 22:32:55.102547 (+ 171us) client_negotiation.cc:240] Negotiation successful
0618 22:32:55.102872 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":309,"threads_started":1}
I20260618 22:32:55.103981 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.089499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:51611 (local address 127.18.150.190:35743)
0618 22:32:55.089669 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:55.089674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:55.090237 (+ 563us) server_negotiation.cc:407] Connection header received
0618 22:32:55.090433 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:55.090438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:55.090506 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:55.090580 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:55.091697 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.092417 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.094343 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.094945 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.095053 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.096922 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:55.096929 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:55.096934 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:55.096973 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:55.100079 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.100946 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.100952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.100955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.101110 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.101518 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.101524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.101527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.101960 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:55.102105 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:55.103591 (+ 1486us) server_negotiation.cc:299] Negotiation successful
0618 22:32:55.103774 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:32:55.106894 20979 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:39121
I20260618 22:32:55.106930 21113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:39121 every 8 connection(s)
I20260618 22:32:55.108043 20979 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:32:55.108508 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 20979
I20260618 22:32:55.108686 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:32:55.109201 19034 external_mini_cluster.cc:1428] Setting key bcd5e121e719d379584c9e3d6dc7535d
I20260618 22:32:55.110127 21114 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35743
I20260618 22:32:55.110339 21114 heartbeater.cc:464] Registering TS with master...
I20260618 22:32:55.111052 21114 heartbeater.cc:511] Master 127.18.150.190:35743 requested a full tablet report, sending...
I20260618 22:32:55.112242 20767 ts_manager.cc:194] Registered new tserver with Master: 76d5a92bac054a028eaf283943f8f500 (127.18.150.130:39121)
I20260618 22:32:55.113104 20767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:52655
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:32:55.145889 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35743
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:33161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:32:55.447088 21118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:32:55.447359 21118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:32:55.447494 21118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:32:55.454550 21118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:32:55.454661 21118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:32:55.454767 21118 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260618 22:32:55.454838 21118 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:32:55.454945 21118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:32:55.464560 21118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:33161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.150.190:35743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:32:55.465804 21118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:32:55.467068 21118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:32:55.479133 21124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:55.479164 21123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:32:55.479096 21126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:32:55.480049 21118 server_base.cc:1034] running on GCE node
I20260618 22:32:55.480670 21118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:32:55.481576 21118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:32:55.482764 21118 hybrid_clock.cc:648] HybridClock initialized: now 1781821975482702 us; error 55 us; skew 500 ppm
I20260618 22:32:55.483047 21118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:32:55 dist-test-slave-h7jz krb5kdc[20723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821975, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:32:55.486047 21118 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:32:55.487533 21118 webserver.cc:460] Webserver started at http://127.18.150.131:45465/ using document root <none> and password file <none>
I20260618 22:32:55.488168 21118 fs_manager.cc:359] Metadata directory not provided
I20260618 22:32:55.488281 21118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:32:55.488580 21118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:32:55.491508 21118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "5d0f526169e44a129e3afce5a1194710"
format_stamp: "Formatted at 2026-06-18 22:32:55 on dist-test-slave-h7jz"
server_key: "d1d65730b95bab400efc3ab76b3f5938"
server_key_iv: "08aa1bcffcfd98b4cb4e14b400391eb2"
server_key_version: "encryptionkey@0"
I20260618 22:32:55.492210 21118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "5d0f526169e44a129e3afce5a1194710"
format_stamp: "Formatted at 2026-06-18 22:32:55 on dist-test-slave-h7jz"
server_key: "d1d65730b95bab400efc3ab76b3f5938"
server_key_iv: "08aa1bcffcfd98b4cb4e14b400391eb2"
server_key_version: "encryptionkey@0"
I20260618 22:32:55.497179 21118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:32:55.500586 21133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:55.501861 21118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:32:55.502059 21118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "5d0f526169e44a129e3afce5a1194710"
format_stamp: "Formatted at 2026-06-18 22:32:55 on dist-test-slave-h7jz"
server_key: "d1d65730b95bab400efc3ab76b3f5938"
server_key_iv: "08aa1bcffcfd98b4cb4e14b400391eb2"
server_key_version: "encryptionkey@0"
I20260618 22:32:55.502226 21118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:32:55.520696 21118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:32:55.524575 21118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:32:55.524828 21118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:32:55.527423 21118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:32:55.527508 21118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:32:55.527635 21118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:32:55.527745 21118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Jun 18 22:32:55 dist-test-slave-h7jz krb5kdc[20723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821975, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:55.549343 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.527820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:50163 (local address 127.18.150.190:35743)
0618 22:32:55.528004 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:55.528012 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:55.529501 (+ 1489us) server_negotiation.cc:407] Connection header received
0618 22:32:55.531482 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:55.531488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:55.531577 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:55.531663 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:55.533661 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.534654 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.536990 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.537402 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.537529 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.542373 (+ 4844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:55.542379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:55.542385 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:55.542427 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:55.545424 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.546434 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.546440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.546444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.546618 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.547180 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.547185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.547188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.547504 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:55.547654 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:55.548882 (+ 1228us) server_negotiation.cc:299] Negotiation successful
0618 22:32:55.549091 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:32:55.551110 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.528437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35743 (local address 127.18.150.131:50163)
0618 22:32:55.529320 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:55.529368 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:55.531212 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:55.531893 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:55.531901 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:55.532593 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:55.533470 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.533501 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.534799 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.534805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.536857 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.536872 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.537797 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.537804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.538080 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.538803 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:55.538844 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:55.542072 (+ 3228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:55.545606 (+ 3534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:55.545616 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:55.545636 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:55.546310 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:55.546754 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:55.546762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:55.546765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:55.547020 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:55.547673 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:55.547682 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:55.548407 (+ 725us) client_negotiation.cc:769] Sending connection context
0618 22:32:55.548752 (+ 345us) client_negotiation.cc:240] Negotiation successful
0618 22:32:55.549370 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":332,"threads_started":1}
I20260618 22:32:55.562572 21118 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:42943
I20260618 22:32:55.562633 21248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:42943 every 8 connection(s)
I20260618 22:32:55.563786 21118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:32:55.565881 21249 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35743
I20260618 22:32:55.566170 21249 heartbeater.cc:464] Registering TS with master...
I20260618 22:32:55.567168 21249 heartbeater.cc:511] Master 127.18.150.190:35743 requested a full tablet report, sending...
I20260618 22:32:55.568753 20767 ts_manager.cc:194] Registered new tserver with Master: 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943)
I20260618 22:32:55.569650 20767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:50163
I20260618 22:32:55.571337 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 21118
I20260618 22:32:55.571494 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:32:55.571988 19034 external_mini_cluster.cc:1428] Setting key fbfc7d1a9371816a24d6109d41157312
I20260618 22:32:55.574530 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:32:55.579291 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.568748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:51295 (local address 127.18.150.190:35743)
0618 22:32:55.568902 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:55.568907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:55.569373 (+ 466us) server_negotiation.cc:407] Connection header received
0618 22:32:55.569559 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:55.569564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:55.569620 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:55.569718 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:55.570632 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.571326 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.572566 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.572977 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.573080 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.574638 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:55.574645 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:55.574650 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:55.574684 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:55.577021 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.577570 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.577576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.577580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.577697 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.578033 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.578038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.578042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.578286 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:55.578416 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:55.578765 (+ 349us) server_negotiation.cc:299] Negotiation successful
0618 22:32:55.578952 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:32:55.579336 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.568798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35743 (local address 127.18.150.131:51295)
0618 22:32:55.569280 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:55.569295 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:55.569413 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:55.569736 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:55.569741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:55.570255 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:55.570510 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.570519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.571439 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.571445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.572444 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.572457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.573131 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.573137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.573285 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.573813 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:55.573832 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:55.574475 (+ 643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:55.577130 (+ 2655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:55.577136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:55.577140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:55.577472 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:55.577790 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:55.577795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:55.577799 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:55.577908 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:55.578388 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:55.578393 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:55.578548 (+ 155us) client_negotiation.cc:769] Sending connection context
0618 22:32:55.578654 (+ 106us) client_negotiation.cc:240] Negotiation successful
0618 22:32:55.578919 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":243,"threads_started":1}
Jun 18 22:32:55 dist-test-slave-h7jz krb5kdc[20723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:32:55 dist-test-slave-h7jz krb5kdc[20723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:55.616947 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.603886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34520 (local address 127.18.150.190:35743)
0618 22:32:55.604041 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:55.604045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:55.604401 (+ 356us) server_negotiation.cc:407] Connection header received
0618 22:32:55.604556 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:55.604560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:55.604624 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:55.604713 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:55.606056 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.606858 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.608191 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.608608 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.608718 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.612427 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:55.612434 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:55.612440 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:55.612479 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:55.614896 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.615454 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.615457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.615460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.615563 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.615856 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.615859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.615861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.616106 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:55.616291 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:55.616487 (+ 196us) server_negotiation.cc:299] Negotiation successful
0618 22:32:55.616665 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:32:55.620869 20767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:32:55.627496 20767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:32:55.637981 20975 heartbeater.cc:503] Master 127.18.150.190:35743 was elected leader, sending a full tablet report...
I20260618 22:32:55.659790 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.639827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33984 (local address 127.18.150.130:39121)
0618 22:32:55.651431 (+ 11604us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:55.651442 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:55.651476 (+ 34us) server_negotiation.cc:407] Connection header received
0618 22:32:55.651588 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:55.651596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:55.651887 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:55.653099 (+ 1212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:55.653303 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.654985 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.657459 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.658568 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.658700 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.658879 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:55.659342 (+ 463us) server_negotiation.cc:299] Negotiation successful
0618 22:32:55.659511 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11417,"thread_start_us":12230,"threads_started":1}
I20260618 22:32:55.659876 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.639330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:39121 (local address 127.0.0.1:33984)
0618 22:32:55.646704 (+ 7374us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:55.646722 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:55.646825 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:55.652381 (+ 5556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:55.652391 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:55.652426 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:55.652867 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.652881 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.655159 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.655165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.657296 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.657309 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.658807 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.658819 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.659045 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.659111 (+ 66us) client_negotiation.cc:769] Sending connection context
0618 22:32:55.659266 (+ 155us) client_negotiation.cc:240] Negotiation successful
0618 22:32:55.659463 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7255,"thread_start_us":6336,"threads_started":1}
I20260618 22:32:55.663838 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.639362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37620 (local address 127.18.150.131:42943)
0618 22:32:55.655528 (+ 16166us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:55.655534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:55.655564 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:32:55.655644 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:55.655650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:55.655840 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:55.656026 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:55.656819 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.658482 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.660826 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.662253 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.662400 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.662580 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:55.663323 (+ 743us) server_negotiation.cc:299] Negotiation successful
0618 22:32:55.663477 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15990,"thread_start_us":15809,"threads_started":1}
I20260618 22:32:55.663861 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.638954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:42943 (local address 127.0.0.1:37620)
0618 22:32:55.646120 (+ 7166us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:55.646161 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:55.646328 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:55.656036 (+ 9708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:55.656041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:55.656066 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:55.656550 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.656565 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.658628 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.658634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.660667 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.660680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.662905 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.662913 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.663110 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.663148 (+ 38us) client_negotiation.cc:769] Sending connection context
0618 22:32:55.663246 (+ 98us) client_negotiation.cc:240] Negotiation successful
0618 22:32:55.663412 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6876,"mutex_wait_us":46,"thread_start_us":7493,"threads_started":1}
I20260618 22:32:55.663975 21044 tablet_service.cc:1467] Processing CreateTablet for tablet 8146014ad3004483ae7a214ea11c2cc4 (DEFAULT_TABLE table=test-table [id=0032082030b9434eb2f35759a208b83a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:55.664673 21270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.647090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41121 (local address 127.0.0.1:34816)
0618 22:32:55.654731 (+ 7641us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:55.654751 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:55.654864 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:55.657397 (+ 2533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:55.657403 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:55.657429 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:55.657851 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.657863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.659834 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.659840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.661930 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.661973 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.663872 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.663879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.664063 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.664099 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:32:55.664182 (+ 83us) client_negotiation.cc:240] Negotiation successful
0618 22:32:55.664297 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6572,"mutex_wait_us":3,"thread_start_us":6665,"threads_started":1}
I20260618 22:32:55.665331 21044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8146014ad3004483ae7a214ea11c2cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:55.665653 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.647188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34816 (local address 127.18.150.129:41121)
0618 22:32:55.656767 (+ 9579us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:55.656776 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:55.656803 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:32:55.656906 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:55.656914 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:55.657152 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:55.658083 (+ 931us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:55.658273 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.659676 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.662151 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.663625 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.665024 (+ 1399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.665192 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:55.665266 (+ 74us) server_negotiation.cc:299] Negotiation successful
0618 22:32:55.665388 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9427,"thread_start_us":9291,"threads_started":1}
I20260618 22:32:55.667774 21183 tablet_service.cc:1467] Processing CreateTablet for tablet 8146014ad3004483ae7a214ea11c2cc4 (DEFAULT_TABLE table=test-table [id=0032082030b9434eb2f35759a208b83a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:55.668644 20905 tablet_service.cc:1467] Processing CreateTablet for tablet 8146014ad3004483ae7a214ea11c2cc4 (DEFAULT_TABLE table=test-table [id=0032082030b9434eb2f35759a208b83a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:55.669149 21183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8146014ad3004483ae7a214ea11c2cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:55.669708 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8146014ad3004483ae7a214ea11c2cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:55.679400 21272 tablet_bootstrap.cc:492] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: Bootstrap starting.
I20260618 22:32:55.681314 21273 tablet_bootstrap.cc:492] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: Bootstrap starting.
I20260618 22:32:55.682823 21274 tablet_bootstrap.cc:492] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: Bootstrap starting.
I20260618 22:32:55.683138 21272 tablet_bootstrap.cc:654] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:55.684504 21272 log.cc:824] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:55.684551 21273 tablet_bootstrap.cc:654] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:55.685738 21273 log.cc:824] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:55.686518 21274 tablet_bootstrap.cc:654] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:55.687588 21274 log.cc:824] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: Log is configured to *not* fsync() on all Append() calls
I20260618 22:32:55.688531 21272 tablet_bootstrap.cc:492] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: No bootstrap required, opened a new log
I20260618 22:32:55.688541 21273 tablet_bootstrap.cc:492] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: No bootstrap required, opened a new log
I20260618 22:32:55.688789 21272 ts_tablet_manager.cc:1397] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260618 22:32:55.688789 21273 ts_tablet_manager.cc:1397] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260618 22:32:55.690044 21274 tablet_bootstrap.cc:492] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: No bootstrap required, opened a new log
I20260618 22:32:55.690292 21274 ts_tablet_manager.cc:1397] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260618 22:32:55.697849 21274 raft_consensus.cc:348] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } }
I20260618 22:32:55.698258 21274 raft_consensus.cc:374] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:55.698405 21274 raft_consensus.cc:729] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d0f526169e44a129e3afce5a1194710, State: Initialized, Role: FOLLOWER
I20260618 22:32:55.698992 21274 consensus_queue.cc:260] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } }
I20260618 22:32:55.698895 21273 raft_consensus.cc:348] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } }
I20260618 22:32:55.698896 21272 raft_consensus.cc:348] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } }
I20260618 22:32:55.699295 21272 raft_consensus.cc:374] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:55.699280 21273 raft_consensus.cc:374] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:55.699378 21273 raft_consensus.cc:729] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b4b5b6f36641c39c5a020413c59c67, State: Initialized, Role: FOLLOWER
I20260618 22:32:55.699431 21272 raft_consensus.cc:729] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d5a92bac054a028eaf283943f8f500, State: Initialized, Role: FOLLOWER
I20260618 22:32:55.700148 21249 heartbeater.cc:503] Master 127.18.150.190:35743 was elected leader, sending a full tablet report...
I20260618 22:32:55.700086 21273 consensus_queue.cc:260] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } }
I20260618 22:32:55.700227 21272 consensus_queue.cc:260] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } }
I20260618 22:32:55.701905 21273 ts_tablet_manager.cc:1428] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: Time spent starting tablet: real 0.013s user 0.006s sys 0.008s
I20260618 22:32:55.702559 21274 ts_tablet_manager.cc:1428] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260618 22:32:55.706929 21114 heartbeater.cc:503] Master 127.18.150.190:35743 was elected leader, sending a full tablet report...
I20260618 22:32:55.707237 21272 ts_tablet_manager.cc:1428] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: Time spent starting tablet: real 0.018s user 0.009s sys 0.008s
I20260618 22:32:55.800431 21278 raft_consensus.cc:482] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:32:55.800822 21278 raft_consensus.cc:504] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } }
I20260618 22:32:55.803032 21278 leader_election.cc:288] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121), 76d5a92bac054a028eaf283943f8f500 (127.18.150.130:39121)
I20260618 22:32:55.810477 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.803793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:60847 (local address 127.18.150.130:39121)
0618 22:32:55.803972 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:55.803978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:55.803998 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:32:55.804071 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:55.804075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:55.804162 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:55.804243 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:55.804663 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.805587 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.808254 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.809436 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.809596 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.809669 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:55.810038 (+ 369us) server_negotiation.cc:299] Negotiation successful
0618 22:32:55.810192 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:32:55.810676 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.803813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:47549 (local address 127.18.150.129:41121)
0618 22:32:55.804009 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:55.804015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:55.804097 (+ 82us) server_negotiation.cc:407] Connection header received
0618 22:32:55.804317 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:55.804321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:55.804391 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:55.804477 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:55.805076 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.806264 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.808493 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.809633 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.809741 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.809803 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:55.810332 (+ 529us) server_negotiation.cc:299] Negotiation successful
0618 22:32:55.810464 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:32:55.810808 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.803374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41121 (local address 127.18.150.131:47549)
0618 22:32:55.804021 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:55.804037 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:55.804173 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:55.804503 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:55.804508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:55.804533 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:55.804874 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.804886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.806386 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.806392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.808330 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.808343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.810022 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.810028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.810191 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.810223 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:32:55.810311 (+ 88us) client_negotiation.cc:240] Negotiation successful
0618 22:32:55.810440 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":20,"thread_start_us":562,"threads_started":1}
I20260618 22:32:55.811481 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.803288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:39121 (local address 127.18.150.131:60847)
0618 22:32:55.803525 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:55.803552 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:55.803702 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:55.804289 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:55.804293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:55.804318 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:55.804521 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.804533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.805708 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.805713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.807759 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.807774 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.809599 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.809606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.809809 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.809846 (+ 37us) client_negotiation.cc:769] Sending connection context
0618 22:32:55.811048 (+ 1202us) client_negotiation.cc:240] Negotiation successful
0618 22:32:55.811199 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":57}
I20260618 22:32:55.811800 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8146014ad3004483ae7a214ea11c2cc4" candidate_uuid: "5d0f526169e44a129e3afce5a1194710" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b4b5b6f36641c39c5a020413c59c67" is_pre_election: true
I20260618 22:32:55.812345 20925 raft_consensus.cc:2454] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d0f526169e44a129e3afce5a1194710 in term 0.
I20260618 22:32:55.813045 21135 leader_election.cc:302] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d0f526169e44a129e3afce5a1194710, b5b4b5b6f36641c39c5a020413c59c67; no voters:
I20260618 22:32:55.813485 21278 raft_consensus.cc:2781] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:32:55.813645 21278 raft_consensus.cc:482] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:32:55.813783 21278 raft_consensus.cc:3037] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:55.813658 21064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8146014ad3004483ae7a214ea11c2cc4" candidate_uuid: "5d0f526169e44a129e3afce5a1194710" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76d5a92bac054a028eaf283943f8f500" is_pre_election: true
I20260618 22:32:55.814173 21064 raft_consensus.cc:2454] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d0f526169e44a129e3afce5a1194710 in term 0.
W20260618 22:32:55.815166 21250 tablet.cc:2369] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:32:55.817116 21278 raft_consensus.cc:504] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } }
I20260618 22:32:55.817974 21278 leader_election.cc:288] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [CANDIDATE]: Term 1 election: Requested vote from peers b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121), 76d5a92bac054a028eaf283943f8f500 (127.18.150.130:39121)
I20260618 22:32:55.818459 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8146014ad3004483ae7a214ea11c2cc4" candidate_uuid: "5d0f526169e44a129e3afce5a1194710" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b4b5b6f36641c39c5a020413c59c67"
I20260618 22:32:55.818480 21064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8146014ad3004483ae7a214ea11c2cc4" candidate_uuid: "5d0f526169e44a129e3afce5a1194710" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76d5a92bac054a028eaf283943f8f500"
I20260618 22:32:55.818645 20925 raft_consensus.cc:3037] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:55.818650 21064 raft_consensus.cc:3037] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:55.820710 20925 raft_consensus.cc:2454] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d0f526169e44a129e3afce5a1194710 in term 1.
I20260618 22:32:55.820724 21064 raft_consensus.cc:2454] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d0f526169e44a129e3afce5a1194710 in term 1.
I20260618 22:32:55.821069 21135 leader_election.cc:302] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d0f526169e44a129e3afce5a1194710, b5b4b5b6f36641c39c5a020413c59c67; no voters:
I20260618 22:32:55.821358 21278 raft_consensus.cc:2781] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:55.821779 21278 raft_consensus.cc:686] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 1 LEADER]: Becoming Leader. State: Replica: 5d0f526169e44a129e3afce5a1194710, State: Running, Role: LEADER
I20260618 22:32:55.822252 21278 consensus_queue.cc:237] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } }
I20260618 22:32:55.827108 20767 catalog_manager.cc:5495] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d0f526169e44a129e3afce5a1194710 (127.18.150.131). New cstate: current_term: 1 leader_uuid: "5d0f526169e44a129e3afce5a1194710" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } health_report { overall_health: UNKNOWN } } }
W20260618 22:32:55.859935 21115 tablet.cc:2369] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:32:55.879812 20976 tablet.cc:2369] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 18 22:32:55 dist-test-slave-h7jz krb5kdc[20723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821973, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:55.888067 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.876155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.18.150.190:35743)
0618 22:32:55.876339 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:55.876348 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:55.876848 (+ 500us) server_negotiation.cc:407] Connection header received
0618 22:32:55.876988 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:55.876992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:55.877047 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:55.877128 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:55.878207 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.878892 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.880048 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.880444 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.880544 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.883588 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:55.883593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:55.883602 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:55.883630 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:55.885755 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.886376 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.886380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.886382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.886561 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:55.886857 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:55.886860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:55.886862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:55.887148 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:55.887245 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:55.887634 (+ 389us) server_negotiation.cc:299] Negotiation successful
0618 22:32:55.887804 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":60}
I20260618 22:32:55.888073 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:55.876217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35743 (local address 127.0.0.1:34530)
0618 22:32:55.876750 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:55.876765 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:55.876888 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:55.877163 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:55.877166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:55.877801 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:32:55.878087 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.878097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.878992 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.878996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.879901 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:55.879911 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:55.880607 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:55.880614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:55.880755 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:55.881336 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:32:55.881356 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:32:55.883437 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:32:55.885870 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:55.885880 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:55.885886 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:55.886293 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:55.886648 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:32:55.886651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:32:55.886653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:32:55.886773 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:32:55.887254 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:32:55.887261 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:32:55.887409 (+ 148us) client_negotiation.cc:769] Sending connection context
0618 22:32:55.887527 (+ 118us) client_negotiation.cc:240] Negotiation successful
0618 22:32:55.887713 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":284,"threads_started":1}
I20260618 22:32:55.892717 20767 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:34530:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260618 22:32:55.893656 20767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:32:55.899788 20905 tablet_service.cc:1467] Processing CreateTablet for tablet 9c9eef04b8f74b0d982a816d701264c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9269d3b58a48431c871052dfb87c077e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:32:55.900048 21044 tablet_service.cc:1467] Processing CreateTablet for tablet 9c9eef04b8f74b0d982a816d701264c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9269d3b58a48431c871052dfb87c077e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:32:55.900025 21183 tablet_service.cc:1467] Processing CreateTablet for tablet 9c9eef04b8f74b0d982a816d701264c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9269d3b58a48431c871052dfb87c077e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:32:55.900803 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c9eef04b8f74b0d982a816d701264c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:55.900874 21044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c9eef04b8f74b0d982a816d701264c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:55.900874 21183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c9eef04b8f74b0d982a816d701264c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:55.907459 21273 tablet_bootstrap.cc:492] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67: Bootstrap starting.
I20260618 22:32:55.907593 21274 tablet_bootstrap.cc:492] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710: Bootstrap starting.
I20260618 22:32:55.908115 21272 tablet_bootstrap.cc:492] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: Bootstrap starting.
I20260618 22:32:55.908855 21274 tablet_bootstrap.cc:654] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:55.908982 21273 tablet_bootstrap.cc:654] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:55.909636 21272 tablet_bootstrap.cc:654] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:55.910827 21274 tablet_bootstrap.cc:492] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710: No bootstrap required, opened a new log
I20260618 22:32:55.910962 21274 ts_tablet_manager.cc:1397] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260618 22:32:55.911037 21273 tablet_bootstrap.cc:492] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67: No bootstrap required, opened a new log
I20260618 22:32:55.911162 21273 ts_tablet_manager.cc:1397] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260618 22:32:55.911707 21274 raft_consensus.cc:348] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:55.911804 21273 raft_consensus.cc:348] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:55.911957 21274 raft_consensus.cc:374] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:55.912088 21274 raft_consensus.cc:729] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d0f526169e44a129e3afce5a1194710, State: Initialized, Role: FOLLOWER
I20260618 22:32:55.912005 21273 raft_consensus.cc:374] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:55.912243 21273 raft_consensus.cc:729] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b4b5b6f36641c39c5a020413c59c67, State: Initialized, Role: FOLLOWER
I20260618 22:32:55.912374 21274 consensus_queue.cc:260] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:55.912454 21273 consensus_queue.cc:260] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:55.912973 21274 ts_tablet_manager.cc:1428] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 22:32:55.912853 21278 tablet_replica.cc:439] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } }
I20260618 22:32:55.912874 21279 tablet_replica.cc:439] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } }
I20260618 22:32:55.913139 21278 tablet_replica.cc:442] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:55.913203 21279 tablet_replica.cc:442] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:55.914165 21273 ts_tablet_manager.cc:1428] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 22:32:55.917063 21272 tablet_bootstrap.cc:492] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: No bootstrap required, opened a new log
I20260618 22:32:55.917217 21272 ts_tablet_manager.cc:1397] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260618 22:32:55.917887 21272 raft_consensus.cc:348] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:55.918157 21272 raft_consensus.cc:374] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:55.918248 21272 raft_consensus.cc:729] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d5a92bac054a028eaf283943f8f500, State: Initialized, Role: FOLLOWER
I20260618 22:32:55.918457 21272 consensus_queue.cc:260] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:55.918958 21272 ts_tablet_manager.cc:1428] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:32:55.918941 21280 tablet_replica.cc:439] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } }
I20260618 22:32:55.919332 21280 tablet_replica.cc:442] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:56.083743 21280 raft_consensus.cc:482] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:32:56.083966 21280 raft_consensus.cc:504] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.085445 21280 leader_election.cc:288] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121), 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943)
I20260618 22:32:56.093127 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.086592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:54921 (local address 127.18.150.131:42943)
0618 22:32:56.086746 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:56.086752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:56.087183 (+ 431us) server_negotiation.cc:407] Connection header received
0618 22:32:56.087332 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:56.087335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:56.087421 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:56.087508 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:56.087970 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.088828 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.091077 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.092214 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.092342 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.092401 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:56.092743 (+ 342us) server_negotiation.cc:299] Negotiation successful
0618 22:32:56.092874 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:32:56.093151 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.085796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:51869 (local address 127.18.150.129:41121)
0618 22:32:56.086013 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:56.086020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:56.086413 (+ 393us) server_negotiation.cc:407] Connection header received
0618 22:32:56.086664 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:56.086670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:56.086752 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:56.086853 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:56.087440 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.088476 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.090904 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.092107 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.092271 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.092342 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:56.092740 (+ 398us) server_negotiation.cc:299] Negotiation successful
0618 22:32:56.092885 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:32:56.093351 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.085740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41121 (local address 127.18.150.130:51869)
0618 22:32:56.086291 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:56.086310 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:56.086501 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:56.086892 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:56.086897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:56.086933 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:56.087281 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.087292 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.088612 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.088616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.090762 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.090777 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.092310 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.092316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.092500 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.092592 (+ 92us) client_negotiation.cc:769] Sending connection context
0618 22:32:56.092686 (+ 94us) client_negotiation.cc:240] Negotiation successful
0618 22:32:56.092880 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":56,"thread_start_us":309,"threads_started":1}
I20260618 22:32:56.093371 21294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.086487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:42943 (local address 127.18.150.130:54921)
0618 22:32:56.087082 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:56.087102 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:56.087220 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:56.087513 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:56.087516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:56.087534 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:56.087818 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.087826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.088955 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.088960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.090919 (+ 1959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.090937 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.092397 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.092402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.092596 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.092621 (+ 25us) client_negotiation.cc:769] Sending connection context
0618 22:32:56.092690 (+ 69us) client_negotiation.cc:240] Negotiation successful
0618 22:32:56.092881 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":73,"thread_start_us":289,"threads_started":1}
I20260618 22:32:56.094116 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9eef04b8f74b0d982a816d701264c5" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b4b5b6f36641c39c5a020413c59c67" is_pre_election: true
I20260618 22:32:56.094347 20925 raft_consensus.cc:2454] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 0.
I20260618 22:32:56.094321 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9eef04b8f74b0d982a816d701264c5" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d0f526169e44a129e3afce5a1194710" is_pre_election: true
I20260618 22:32:56.094693 21203 raft_consensus.cc:2454] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 0.
I20260618 22:32:56.094770 20996 leader_election.cc:302] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76d5a92bac054a028eaf283943f8f500, b5b4b5b6f36641c39c5a020413c59c67; no voters:
I20260618 22:32:56.095175 21280 raft_consensus.cc:2781] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:32:56.095299 21280 raft_consensus.cc:482] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:32:56.095400 21280 raft_consensus.cc:3037] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:56.097379 21280 raft_consensus.cc:504] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.097908 21280 leader_election.cc:288] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 election: Requested vote from peers b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121), 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943)
I20260618 22:32:56.098338 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9eef04b8f74b0d982a816d701264c5" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b4b5b6f36641c39c5a020413c59c67"
I20260618 22:32:56.098448 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c9eef04b8f74b0d982a816d701264c5" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d0f526169e44a129e3afce5a1194710"
I20260618 22:32:56.098532 20925 raft_consensus.cc:3037] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:56.098628 21203 raft_consensus.cc:3037] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:56.100466 20925 raft_consensus.cc:2454] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 1.
I20260618 22:32:56.100545 21203 raft_consensus.cc:2454] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 1.
I20260618 22:32:56.100876 20996 leader_election.cc:302] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d0f526169e44a129e3afce5a1194710, 76d5a92bac054a028eaf283943f8f500; no voters:
I20260618 22:32:56.101167 21280 raft_consensus.cc:2781] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:56.101552 21280 raft_consensus.cc:686] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [term 1 LEADER]: Becoming Leader. State: Replica: 76d5a92bac054a028eaf283943f8f500, State: Running, Role: LEADER
I20260618 22:32:56.102041 21280 consensus_queue.cc:237] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.105233 21295 tablet_replica.cc:439] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: TxnStatusTablet state changed. Reason: New leader 76d5a92bac054a028eaf283943f8f500. Latest consensus state: current_term: 1 leader_uuid: "76d5a92bac054a028eaf283943f8f500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } }
I20260618 22:32:56.105608 21295 tablet_replica.cc:442] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:32:56.106443 21297 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260618 22:32:56.106678 21297 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260618 22:32:56.111101 20768 catalog_manager.cc:5495] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76d5a92bac054a028eaf283943f8f500 (127.18.150.130). New cstate: current_term: 1 leader_uuid: "76d5a92bac054a028eaf283943f8f500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } health_report { overall_health: UNKNOWN } } }
I20260618 22:32:56.151172 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.145825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:39121 (local address 127.0.0.1:33986)
0618 22:32:56.146086 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:56.146107 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:56.146234 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:56.146672 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:56.146676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:56.146688 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:32:56.146841 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.146850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.147860 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.147865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.148834 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.148845 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.149527 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.149533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.149689 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.149746 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.150606 (+ 860us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.150626 (+ 20us) client_negotiation.cc:769] Sending connection context
0618 22:32:56.150707 (+ 81us) client_negotiation.cc:240] Negotiation successful
0618 22:32:56.150857 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260618 22:32:56.151203 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.145972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.18.150.130:39121)
0618 22:32:56.146137 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:56.146143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:56.146161 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:32:56.146435 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:56.146439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:56.146519 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:56.146646 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:56.146960 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.147736 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.148962 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.149376 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.149502 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.149915 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.150506 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.150641 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:56.150737 (+ 96us) server_negotiation.cc:299] Negotiation successful
0618 22:32:56.150821 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:32:56.159584 20925 raft_consensus.cc:1261] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Refusing update from remote peer 76d5a92bac054a028eaf283943f8f500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:56.159569 21203 raft_consensus.cc:1261] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Refusing update from remote peer 76d5a92bac054a028eaf283943f8f500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:56.160506 21295 consensus_queue.cc:1035] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:56.160933 21280 consensus_queue.cc:1035] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:56.170626 21278 tablet_replica.cc:439] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710: TxnStatusTablet state changed. Reason: New leader 76d5a92bac054a028eaf283943f8f500. Latest consensus state: current_term: 1 leader_uuid: "76d5a92bac054a028eaf283943f8f500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } }
I20260618 22:32:56.170909 21278 tablet_replica.cc:442] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:56.172703 21300 mvcc.cc:205] Tried to move back new op lower bound from 7298342814339919872 to 7298342814117871616. Current Snapshot: MvccSnapshot[applied={T|T < 7298342814339919872}]
I20260618 22:32:56.173887 21301 mvcc.cc:205] Tried to move back new op lower bound from 7298342814339919872 to 7298342814117871616. Current Snapshot: MvccSnapshot[applied={T|T < 7298342814339919872}]
I20260618 22:32:56.176751 21295 tablet_replica.cc:439] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "76d5a92bac054a028eaf283943f8f500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } }
I20260618 22:32:56.176996 21295 tablet_replica.cc:442] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:32:56.181634 21279 tablet_replica.cc:439] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67: TxnStatusTablet state changed. Reason: New leader 76d5a92bac054a028eaf283943f8f500. Latest consensus state: current_term: 1 leader_uuid: "76d5a92bac054a028eaf283943f8f500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } }
I20260618 22:32:56.181918 21279 tablet_replica.cc:442] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:56.183215 21280 tablet_replica.cc:439] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "76d5a92bac054a028eaf283943f8f500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } }
I20260618 22:32:56.183493 21280 tablet_replica.cc:442] T 9c9eef04b8f74b0d982a816d701264c5 P 76d5a92bac054a028eaf283943f8f500: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:32:56.185204 21278 tablet_replica.cc:439] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "76d5a92bac054a028eaf283943f8f500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } }
I20260618 22:32:56.185453 21278 tablet_replica.cc:442] T 9c9eef04b8f74b0d982a816d701264c5 P 5d0f526169e44a129e3afce5a1194710: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:56.190397 21279 tablet_replica.cc:439] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "76d5a92bac054a028eaf283943f8f500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } } }
I20260618 22:32:56.190634 21279 tablet_replica.cc:442] T 9c9eef04b8f74b0d982a816d701264c5 P b5b4b5b6f36641c39c5a020413c59c67: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:32:56.198933 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.190978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37634 (local address 127.18.150.131:42943)
0618 22:32:56.191145 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:56.191151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:56.193226 (+ 2075us) server_negotiation.cc:407] Connection header received
0618 22:32:56.193391 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:56.193398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:56.193509 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:56.193658 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:56.194018 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.195117 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.196523 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.197134 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.197257 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.197641 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.198336 (+ 695us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.198489 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:56.198536 (+ 47us) server_negotiation.cc:299] Negotiation successful
0618 22:32:56.198619 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:32:56.211572 21278 consensus_queue.cc:1035] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:56.216707 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.209106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:47219 (local address 127.18.150.130:39121)
0618 22:32:56.209282 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:56.209288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:56.209566 (+ 278us) server_negotiation.cc:407] Connection header received
0618 22:32:56.209862 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:56.209867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:56.209971 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:56.210058 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:56.210615 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.211453 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.213463 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.214131 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.214252 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.215921 (+ 1669us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.216049 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.216107 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:56.216316 (+ 209us) server_negotiation.cc:299] Negotiation successful
0618 22:32:56.216399 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260618 22:32:56.217056 21282 consensus_queue.cc:1035] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:56.217490 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.208944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:39121 (local address 127.18.150.131:47219)
0618 22:32:56.209465 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:56.209489 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:56.209649 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:56.210079 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:56.210084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:56.210097 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:32:56.210456 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.210467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.211553 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.211559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.213292 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.213308 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.215409 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.215416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.215621 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.215691 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.216180 (+ 489us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.216190 (+ 10us) client_negotiation.cc:769] Sending connection context
0618 22:32:56.216866 (+ 676us) client_negotiation.cc:240] Negotiation successful
0618 22:32:56.217016 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":233,"threads_started":1}
I20260618 22:32:56.269863 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.263483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:44307 (local address 127.18.150.131:42943)
0618 22:32:56.263653 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:56.263659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:56.264286 (+ 627us) server_negotiation.cc:407] Connection header received
0618 22:32:56.264457 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:56.264463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:56.264547 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:56.264645 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:56.265398 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.266537 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.267966 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.268623 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.268750 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.269146 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.269277 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.269364 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:56.269490 (+ 126us) server_negotiation.cc:299] Negotiation successful
0618 22:32:56.269572 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:32:56.270382 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.263451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:42943 (local address 127.18.150.130:44307)
0618 22:32:56.264189 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:56.264205 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:56.264324 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:56.264831 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:56.264835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:56.264848 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:32:56.265236 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.265249 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.266654 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.266658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.267829 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.267841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.268783 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.268787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.268907 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.268968 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.269366 (+ 398us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.269378 (+ 12us) client_negotiation.cc:769] Sending connection context
0618 22:32:56.269460 (+ 82us) client_negotiation.cc:240] Negotiation successful
0618 22:32:56.269973 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":383,"threads_started":1}
I20260618 22:32:56.310860 20768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:32:56.311141 20768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:32:56.314931 20768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752: Sending DeleteTablet for 3 replicas of tablet 8146014ad3004483ae7a214ea11c2cc4
I20260618 22:32:56.316109 20905 tablet_service.cc:1514] Processing DeleteTablet for tablet 8146014ad3004483ae7a214ea11c2cc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:56 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:34816
I20260618 22:32:56.316443 21044 tablet_service.cc:1514] Processing DeleteTablet for tablet 8146014ad3004483ae7a214ea11c2cc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:56 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:33984
I20260618 22:32:56.317013 21183 tablet_service.cc:1514] Processing DeleteTablet for tablet 8146014ad3004483ae7a214ea11c2cc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:56 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:37620
I20260618 22:32:56.317518 21322 tablet_replica.cc:330] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: stopping tablet replica
I20260618 22:32:56.317986 21322 raft_consensus.cc:2229] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:32:56.318384 21322 raft_consensus.cc:2258] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:56.318668 20768 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:32:56.319489 20768 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:32:56.326857 21043 tablet_service.cc:1467] Processing CreateTablet for tablet f1710fdc54144ee8891643064a837b3c (DEFAULT_TABLE table=test-table [id=26e0b2798ba44b16a7c863335bb380b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:56.327158 20905 tablet_service.cc:1467] Processing CreateTablet for tablet f1710fdc54144ee8891643064a837b3c (DEFAULT_TABLE table=test-table [id=26e0b2798ba44b16a7c863335bb380b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:56.327436 21043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1710fdc54144ee8891643064a837b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:56.327571 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1710fdc54144ee8891643064a837b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:56.328020 21322 ts_tablet_manager.cc:1905] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:56.335161 21272 tablet_bootstrap.cc:492] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500: Bootstrap starting.
I20260618 22:32:56.336292 21273 tablet_bootstrap.cc:492] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67: Bootstrap starting.
I20260618 22:32:56.337007 21272 tablet_bootstrap.cc:654] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:56.337801 21273 tablet_bootstrap.cc:654] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:56.342088 21182 tablet_service.cc:1467] Processing CreateTablet for tablet f1710fdc54144ee8891643064a837b3c (DEFAULT_TABLE table=test-table [id=26e0b2798ba44b16a7c863335bb380b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:56.342532 21182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1710fdc54144ee8891643064a837b3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:56.342638 21323 tablet_replica.cc:330] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: stopping tablet replica
I20260618 22:32:56.343112 21323 raft_consensus.cc:2229] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:32:56.343832 21323 raft_consensus.cc:2258] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:56.346657 21323 ts_tablet_manager.cc:1905] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:56.348381 21274 tablet_bootstrap.cc:492] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710: Bootstrap starting.
I20260618 22:32:56.349258 21273 tablet_bootstrap.cc:492] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67: No bootstrap required, opened a new log
I20260618 22:32:56.349406 21273 ts_tablet_manager.cc:1397] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260618 22:32:56.349658 21322 ts_tablet_manager.cc:1918] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:32:56.349802 21322 log.cc:1196] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/8146014ad3004483ae7a214ea11c2cc4
I20260618 22:32:56.349996 21274 tablet_bootstrap.cc:654] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:56.350186 21273 raft_consensus.cc:348] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.350423 21273 raft_consensus.cc:374] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:56.350499 21273 raft_consensus.cc:729] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b4b5b6f36641c39c5a020413c59c67, State: Initialized, Role: FOLLOWER
I20260618 22:32:56.350770 21273 consensus_queue.cc:260] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.351485 21273 ts_tablet_manager.cc:1428] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:32:56.352958 21324 tablet_replica.cc:330] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: stopping tablet replica
I20260618 22:32:56.353730 21324 raft_consensus.cc:2229] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:32:56.354148 21324 raft_consensus.cc:2258] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:56.356207 21324 ts_tablet_manager.cc:1905] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:56.357856 21323 ts_tablet_manager.cc:1918] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:32:56.358071 21323 log.cc:1196] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/8146014ad3004483ae7a214ea11c2cc4
I20260618 22:32:56.358445 21322 ts_tablet_manager.cc:1939] T 8146014ad3004483ae7a214ea11c2cc4 P b5b4b5b6f36641c39c5a020413c59c67: Deleting consensus metadata
I20260618 22:32:56.358618 21323 ts_tablet_manager.cc:1939] T 8146014ad3004483ae7a214ea11c2cc4 P 5d0f526169e44a129e3afce5a1194710: Deleting consensus metadata
I20260618 22:32:56.360179 20753 catalog_manager.cc:4841] TS 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943): tablet 8146014ad3004483ae7a214ea11c2cc4 (table test-table [id=0032082030b9434eb2f35759a208b83a]) successfully deleted
I20260618 22:32:56.360452 20753 catalog_manager.cc:4841] TS b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121): tablet 8146014ad3004483ae7a214ea11c2cc4 (table test-table [id=0032082030b9434eb2f35759a208b83a]) successfully deleted
I20260618 22:32:56.363644 21324 ts_tablet_manager.cc:1918] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:32:56.363804 21324 log.cc:1196] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/8146014ad3004483ae7a214ea11c2cc4
I20260618 22:32:56.364320 21324 ts_tablet_manager.cc:1939] T 8146014ad3004483ae7a214ea11c2cc4 P 76d5a92bac054a028eaf283943f8f500: Deleting consensus metadata
I20260618 22:32:56.365931 20754 catalog_manager.cc:4841] TS 76d5a92bac054a028eaf283943f8f500 (127.18.150.130:39121): tablet 8146014ad3004483ae7a214ea11c2cc4 (table test-table [id=0032082030b9434eb2f35759a208b83a]) successfully deleted
I20260618 22:32:56.371012 21274 tablet_bootstrap.cc:492] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710: No bootstrap required, opened a new log
I20260618 22:32:56.371171 21274 ts_tablet_manager.cc:1397] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.005s
I20260618 22:32:56.372025 21274 raft_consensus.cc:348] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.372175 21272 tablet_bootstrap.cc:492] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500: No bootstrap required, opened a new log
I20260618 22:32:56.372325 21272 ts_tablet_manager.cc:1397] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500: Time spent bootstrapping tablet: real 0.037s user 0.010s sys 0.002s
I20260618 22:32:56.372342 21274 raft_consensus.cc:374] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:56.372462 21274 raft_consensus.cc:729] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d0f526169e44a129e3afce5a1194710, State: Initialized, Role: FOLLOWER
I20260618 22:32:56.372790 21274 consensus_queue.cc:260] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.373106 21272 raft_consensus.cc:348] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.373389 21272 raft_consensus.cc:374] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:56.373492 21272 raft_consensus.cc:729] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d5a92bac054a028eaf283943f8f500, State: Initialized, Role: FOLLOWER
I20260618 22:32:56.373765 21272 consensus_queue.cc:260] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.374396 21272 ts_tablet_manager.cc:1428] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 22:32:56.375576 21274 ts_tablet_manager.cc:1428] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260618 22:32:56.747910 21279 raft_consensus.cc:482] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:32:56.748123 21279 raft_consensus.cc:504] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.749578 21279 leader_election.cc:288] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76d5a92bac054a028eaf283943f8f500 (127.18.150.130:39121), 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943)
I20260618 22:32:56.761823 21330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.750155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:58341 (local address 127.18.150.130:39121)
0618 22:32:56.756680 (+ 6525us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:56.756686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:56.756705 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:32:56.756773 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:56.756778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:56.756874 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:56.756972 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:56.757639 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.758552 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.760075 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.760814 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.760960 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.761037 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:56.761372 (+ 335us) server_negotiation.cc:299] Negotiation successful
0618 22:32:56.761505 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6402,"thread_start_us":6551,"threads_started":1}
I20260618 22:32:56.761886 21328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.749787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:39121 (local address 127.18.150.129:58341)
0618 22:32:56.756418 (+ 6631us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:56.756442 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:56.756617 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:56.757152 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:56.757157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:56.757200 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:56.757494 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.757505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.758669 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.758674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.759926 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.759940 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.761023 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.761029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.761181 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.761204 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:32:56.761331 (+ 127us) client_negotiation.cc:240] Negotiation successful
0618 22:32:56.761493 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6520,"thread_start_us":6403,"threads_started":1}
I20260618 22:32:56.762709 21064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1710fdc54144ee8891643064a837b3c" candidate_uuid: "b5b4b5b6f36641c39c5a020413c59c67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76d5a92bac054a028eaf283943f8f500" is_pre_election: true
I20260618 22:32:56.762950 21064 raft_consensus.cc:2454] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5b4b5b6f36641c39c5a020413c59c67 in term 0.
I20260618 22:32:56.763478 20857 leader_election.cc:302] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76d5a92bac054a028eaf283943f8f500, b5b4b5b6f36641c39c5a020413c59c67; no voters:
I20260618 22:32:56.763908 21279 raft_consensus.cc:2781] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:32:56.764038 21279 raft_consensus.cc:482] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:32:56.764148 21279 raft_consensus.cc:3037] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:56.764204 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.750133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:41407 (local address 127.18.150.131:42943)
0618 22:32:56.750302 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:56.750309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:56.757370 (+ 7061us) server_negotiation.cc:407] Connection header received
0618 22:32:56.757522 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:56.757528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:56.757626 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:56.757745 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:32:56.758322 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.759447 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.761883 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.763233 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.763383 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.763455 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:56.763801 (+ 346us) server_negotiation.cc:299] Negotiation successful
0618 22:32:56.763920 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:32:56.764773 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.749933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:42943 (local address 127.18.150.129:41407)
0618 22:32:56.757244 (+ 7311us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:56.757266 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:56.757382 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:56.757759 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:56.757763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:56.757788 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:32:56.758143 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.758154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.759637 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.759643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.761711 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.761730 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.763441 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.763448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.763611 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.763645 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:32:56.764372 (+ 727us) client_negotiation.cc:240] Negotiation successful
0618 22:32:56.764507 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7150,"thread_start_us":6934,"threads_started":1}
I20260618 22:32:56.765336 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1710fdc54144ee8891643064a837b3c" candidate_uuid: "b5b4b5b6f36641c39c5a020413c59c67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d0f526169e44a129e3afce5a1194710" is_pre_election: true
I20260618 22:32:56.765537 21203 raft_consensus.cc:2454] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5b4b5b6f36641c39c5a020413c59c67 in term 0.
I20260618 22:32:56.767230 21279 raft_consensus.cc:504] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.767940 21279 leader_election.cc:288] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [CANDIDATE]: Term 1 election: Requested vote from peers 76d5a92bac054a028eaf283943f8f500 (127.18.150.130:39121), 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943)
I20260618 22:32:56.768392 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1710fdc54144ee8891643064a837b3c" candidate_uuid: "b5b4b5b6f36641c39c5a020413c59c67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d0f526169e44a129e3afce5a1194710"
I20260618 22:32:56.768380 21064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1710fdc54144ee8891643064a837b3c" candidate_uuid: "b5b4b5b6f36641c39c5a020413c59c67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76d5a92bac054a028eaf283943f8f500"
I20260618 22:32:56.768575 21203 raft_consensus.cc:3037] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:56.768606 21064 raft_consensus.cc:3037] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:56.770617 21064 raft_consensus.cc:2454] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5b4b5b6f36641c39c5a020413c59c67 in term 1.
I20260618 22:32:56.770953 20857 leader_election.cc:302] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76d5a92bac054a028eaf283943f8f500, b5b4b5b6f36641c39c5a020413c59c67; no voters:
I20260618 22:32:56.771198 21279 raft_consensus.cc:2781] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:56.771548 21203 raft_consensus.cc:2454] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5b4b5b6f36641c39c5a020413c59c67 in term 1.
I20260618 22:32:56.771651 21279 raft_consensus.cc:686] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 1 LEADER]: Becoming Leader. State: Replica: b5b4b5b6f36641c39c5a020413c59c67, State: Running, Role: LEADER
I20260618 22:32:56.772104 21279 consensus_queue.cc:237] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:56.775527 20768 catalog_manager.cc:5495] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129). New cstate: current_term: 1 leader_uuid: "b5b4b5b6f36641c39c5a020413c59c67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } health_report { overall_health: UNKNOWN } } }
I20260618 22:32:56.846613 21342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.814275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34818 (local address 127.18.150.129:41121)
0618 22:32:56.837625 (+ 23350us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:56.837632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:56.837660 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:32:56.837782 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:56.837788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:56.837904 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:56.838129 (+ 225us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:56.839111 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.840454 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.842670 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.843578 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.843700 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.844452 (+ 752us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.845301 (+ 849us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.845444 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:56.846098 (+ 654us) server_negotiation.cc:299] Negotiation successful
0618 22:32:56.846191 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23172,"thread_start_us":29031,"threads_started":1}
I20260618 22:32:56.864045 21330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.858740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:37717 (local address 127.18.150.130:39121)
0618 22:32:56.858921 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:56.858926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:56.859228 (+ 302us) server_negotiation.cc:407] Connection header received
0618 22:32:56.859389 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:56.859393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:56.859467 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:56.859540 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:56.859884 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.860556 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.862453 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.862913 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.863016 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.863467 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.863566 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.863632 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:56.863743 (+ 111us) server_negotiation.cc:299] Negotiation successful
0618 22:32:56.863805 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:32:56.864192 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.858653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:39121 (local address 127.18.150.129:37717)
0618 22:32:56.859133 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:56.859152 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:56.859287 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:56.859559 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:56.859563 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:56.859575 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:32:56.859775 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.859785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.860664 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.860670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.862317 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.862331 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.863061 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.863068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.863245 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.863309 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.863651 (+ 342us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.863660 (+ 9us) client_negotiation.cc:769] Sending connection context
0618 22:32:56.863728 (+ 68us) client_negotiation.cc:240] Negotiation successful
0618 22:32:56.863861 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":209,"threads_started":1}
I20260618 22:32:56.872872 21203 raft_consensus.cc:1261] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Refusing update from remote peer b5b4b5b6f36641c39c5a020413c59c67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:56.872866 21064 raft_consensus.cc:1261] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Refusing update from remote peer b5b4b5b6f36641c39c5a020413c59c67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:56.873519 21331 consensus_queue.cc:1035] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:56.873839 21279 consensus_queue.cc:1035] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:56.879230 21336 mvcc.cc:205] Tried to move back new op lower bound from 7298342817265577984 to 7298342816862191616. Current Snapshot: MvccSnapshot[applied={T|T < 7298342817265577984}]
I20260618 22:32:56.899061 21342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.894349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:43857 (local address 127.18.150.129:41121)
0618 22:32:56.894516 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:56.894522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:56.894977 (+ 455us) server_negotiation.cc:407] Connection header received
0618 22:32:56.895146 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:56.895152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:56.895216 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:56.895286 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:56.895624 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.896330 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.897539 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.897985 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.898097 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.898434 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.898535 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.898605 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:56.898721 (+ 116us) server_negotiation.cc:299] Negotiation successful
0618 22:32:56.898793 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:32:56.899130 21352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:56.894216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41121 (local address 127.18.150.130:43857)
0618 22:32:56.894887 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:56.894902 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:56.895016 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:56.895307 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:56.895311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:56.895322 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:32:56.895509 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.895517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.896456 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.896462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.897408 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:56.897420 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:56.898114 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:56.898120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:56.898268 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:56.898301 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:32:56.898619 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:32:56.898630 (+ 11us) client_negotiation.cc:769] Sending connection context
0618 22:32:56.898694 (+ 64us) client_negotiation.cc:240] Negotiation successful
0618 22:32:56.898824 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":440,"threads_started":1}
W20260618 22:32:56.899997 21336 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260618 22:32:58.504452 20768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:32:58.504693 20768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:32:58.508504 20768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752: Sending DeleteTablet for 3 replicas of tablet f1710fdc54144ee8891643064a837b3c
I20260618 22:32:58.509181 20905 tablet_service.cc:1514] Processing DeleteTablet for tablet f1710fdc54144ee8891643064a837b3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:58 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:34816
I20260618 22:32:58.509320 21044 tablet_service.cc:1514] Processing DeleteTablet for tablet f1710fdc54144ee8891643064a837b3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:58 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:33984
I20260618 22:32:58.509369 21182 tablet_service.cc:1514] Processing DeleteTablet for tablet f1710fdc54144ee8891643064a837b3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:58 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:37620
I20260618 22:32:58.524775 21378 tablet_replica.cc:330] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500: stopping tablet replica
I20260618 22:32:58.524796 21379 tablet_replica.cc:330] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710: stopping tablet replica
I20260618 22:32:58.524992 21378 raft_consensus.cc:2229] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:32:58.524987 21379 raft_consensus.cc:2229] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:32:58.525223 21378 raft_consensus.cc:2258] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:58.525223 21379 raft_consensus.cc:2258] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:58.526099 21377 tablet_replica.cc:330] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67: stopping tablet replica
I20260618 22:32:58.526219 21379 ts_tablet_manager.cc:1905] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:58.526324 21378 ts_tablet_manager.cc:1905] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:58.526342 21377 raft_consensus.cc:2229] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:32:58.526904 21377 raft_consensus.cc:2258] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:58.527894 21377 ts_tablet_manager.cc:1905] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:58.533022 21378 ts_tablet_manager.cc:1918] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:32:58.533146 21379 ts_tablet_manager.cc:1918] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:32:58.533227 21378 log.cc:1196] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/f1710fdc54144ee8891643064a837b3c
I20260618 22:32:58.533315 21379 log.cc:1196] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/f1710fdc54144ee8891643064a837b3c
I20260618 22:32:58.533638 21378 ts_tablet_manager.cc:1939] T f1710fdc54144ee8891643064a837b3c P 76d5a92bac054a028eaf283943f8f500: Deleting consensus metadata
I20260618 22:32:58.533772 21379 ts_tablet_manager.cc:1939] T f1710fdc54144ee8891643064a837b3c P 5d0f526169e44a129e3afce5a1194710: Deleting consensus metadata
I20260618 22:32:58.534808 20754 catalog_manager.cc:4841] TS 76d5a92bac054a028eaf283943f8f500 (127.18.150.130:39121): tablet f1710fdc54144ee8891643064a837b3c (table test-table [id=26e0b2798ba44b16a7c863335bb380b7]) successfully deleted
I20260618 22:32:58.534833 20753 catalog_manager.cc:4841] TS 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943): tablet f1710fdc54144ee8891643064a837b3c (table test-table [id=26e0b2798ba44b16a7c863335bb380b7]) successfully deleted
I20260618 22:32:58.535243 21377 ts_tablet_manager.cc:1918] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:32:58.535374 21377 log.cc:1196] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/f1710fdc54144ee8891643064a837b3c
I20260618 22:32:58.535791 21377 ts_tablet_manager.cc:1939] T f1710fdc54144ee8891643064a837b3c P b5b4b5b6f36641c39c5a020413c59c67: Deleting consensus metadata
I20260618 22:32:58.536814 20753 catalog_manager.cc:4841] TS b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121): tablet f1710fdc54144ee8891643064a837b3c (table test-table [id=26e0b2798ba44b16a7c863335bb380b7]) successfully deleted
Jun 18 22:32:58 dist-test-slave-h7jz krb5kdc[20723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821978, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:32:58 dist-test-slave-h7jz krb5kdc[20723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821978, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:32:58.573365 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:58.551141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.18.150.190:35743)
0618 22:32:58.552009 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:58.552013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:58.552032 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:32:58.552081 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:58.552085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:58.552159 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:58.552253 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:32:58.553579 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:58.554393 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:58.555580 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:58.556030 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:58.556136 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:58.567559 (+ 11423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:32:58.567567 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:32:58.567571 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:32:58.567620 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:32:58.570826 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:58.571476 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:58.571481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:58.571485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:58.571629 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:32:58.571919 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:32:58.571924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:32:58.571927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:32:58.572255 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:32:58.572484 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:58.572770 (+ 286us) server_negotiation.cc:299] Negotiation successful
0618 22:32:58.572997 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":561,"threads_started":1}
I20260618 22:32:58.576858 20768 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:32:58.577517 20768 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:32:58.583791 20905 tablet_service.cc:1467] Processing CreateTablet for tablet 010a4028bb3a4fc0bbcdc918a32bba7b (DEFAULT_TABLE table=test-table [id=61d0f3e152c44adf81e982477565ce1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:58.584004 21044 tablet_service.cc:1467] Processing CreateTablet for tablet 010a4028bb3a4fc0bbcdc918a32bba7b (DEFAULT_TABLE table=test-table [id=61d0f3e152c44adf81e982477565ce1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:58.584111 21182 tablet_service.cc:1467] Processing CreateTablet for tablet 010a4028bb3a4fc0bbcdc918a32bba7b (DEFAULT_TABLE table=test-table [id=61d0f3e152c44adf81e982477565ce1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:58.584273 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010a4028bb3a4fc0bbcdc918a32bba7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:58.584396 21044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010a4028bb3a4fc0bbcdc918a32bba7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:58.584518 21182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010a4028bb3a4fc0bbcdc918a32bba7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:58.590947 21389 tablet_bootstrap.cc:492] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500: Bootstrap starting.
I20260618 22:32:58.592041 21388 tablet_bootstrap.cc:492] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710: Bootstrap starting.
I20260618 22:32:58.592708 21390 tablet_bootstrap.cc:492] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67: Bootstrap starting.
I20260618 22:32:58.593276 21389 tablet_bootstrap.cc:654] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:58.593662 21388 tablet_bootstrap.cc:654] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:58.594329 21390 tablet_bootstrap.cc:654] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:58.595353 21389 tablet_bootstrap.cc:492] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500: No bootstrap required, opened a new log
I20260618 22:32:58.595507 21389 ts_tablet_manager.cc:1397] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260618 22:32:58.595661 21388 tablet_bootstrap.cc:492] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710: No bootstrap required, opened a new log
I20260618 22:32:58.595795 21388 ts_tablet_manager.cc:1397] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:32:58.596168 21390 tablet_bootstrap.cc:492] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67: No bootstrap required, opened a new log
I20260618 22:32:58.596293 21390 ts_tablet_manager.cc:1397] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:32:58.596287 21389 raft_consensus.cc:348] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.596556 21389 raft_consensus.cc:374] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:58.596647 21389 raft_consensus.cc:729] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d5a92bac054a028eaf283943f8f500, State: Initialized, Role: FOLLOWER
I20260618 22:32:58.596485 21388 raft_consensus.cc:348] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.596971 21388 raft_consensus.cc:374] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:58.596868 21389 consensus_queue.cc:260] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.597075 21388 raft_consensus.cc:729] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d0f526169e44a129e3afce5a1194710, State: Initialized, Role: FOLLOWER
I20260618 22:32:58.596989 21390 raft_consensus.cc:348] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.597200 21390 raft_consensus.cc:374] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:58.597288 21390 raft_consensus.cc:729] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b4b5b6f36641c39c5a020413c59c67, State: Initialized, Role: FOLLOWER
I20260618 22:32:58.597414 21389 ts_tablet_manager.cc:1428] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:32:58.597276 21388 consensus_queue.cc:260] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.597528 21390 consensus_queue.cc:260] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.598047 21390 ts_tablet_manager.cc:1428] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:32:58.602440 21388 ts_tablet_manager.cc:1428] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260618 22:32:58.692126 21364 raft_consensus.cc:482] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:32:58.692324 21364 raft_consensus.cc:504] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.692961 21364 leader_election.cc:288] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121), 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943)
I20260618 22:32:58.693450 20924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010a4028bb3a4fc0bbcdc918a32bba7b" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b4b5b6f36641c39c5a020413c59c67" is_pre_election: true
I20260618 22:32:58.693552 21202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010a4028bb3a4fc0bbcdc918a32bba7b" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d0f526169e44a129e3afce5a1194710" is_pre_election: true
I20260618 22:32:58.693665 20924 raft_consensus.cc:2454] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 0.
I20260618 22:32:58.693835 21202 raft_consensus.cc:2454] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 0.
I20260618 22:32:58.694114 20996 leader_election.cc:302] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76d5a92bac054a028eaf283943f8f500, b5b4b5b6f36641c39c5a020413c59c67; no voters:
I20260618 22:32:58.694371 21364 raft_consensus.cc:2781] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:32:58.694486 21364 raft_consensus.cc:482] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:32:58.694582 21364 raft_consensus.cc:3037] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:58.696707 21364 raft_consensus.cc:504] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.697247 21364 leader_election.cc:288] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 election: Requested vote from peers b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121), 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943)
I20260618 22:32:58.697654 20924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010a4028bb3a4fc0bbcdc918a32bba7b" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b4b5b6f36641c39c5a020413c59c67"
I20260618 22:32:58.697762 21202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010a4028bb3a4fc0bbcdc918a32bba7b" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d0f526169e44a129e3afce5a1194710"
I20260618 22:32:58.697860 20924 raft_consensus.cc:3037] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:58.697963 21202 raft_consensus.cc:3037] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:58.700707 21202 raft_consensus.cc:2454] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 1.
I20260618 22:32:58.700709 20924 raft_consensus.cc:2454] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 1.
I20260618 22:32:58.701071 20996 leader_election.cc:302] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76d5a92bac054a028eaf283943f8f500, b5b4b5b6f36641c39c5a020413c59c67; no voters:
I20260618 22:32:58.701323 21364 raft_consensus.cc:2781] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:58.701455 21364 raft_consensus.cc:686] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 1 LEADER]: Becoming Leader. State: Replica: 76d5a92bac054a028eaf283943f8f500, State: Running, Role: LEADER
I20260618 22:32:58.701694 21364 consensus_queue.cc:237] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.704196 20768 catalog_manager.cc:5495] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76d5a92bac054a028eaf283943f8f500 (127.18.150.130). New cstate: current_term: 1 leader_uuid: "76d5a92bac054a028eaf283943f8f500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } health_report { overall_health: UNKNOWN } } }
I20260618 22:32:58.724642 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:58.719197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33994 (local address 127.18.150.130:39121)
0618 22:32:58.719734 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:58.719739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:58.719760 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:32:58.719818 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:58.719823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:58.719893 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:58.720412 (+ 519us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:58.720604 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:58.721645 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:58.723034 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:58.723426 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:58.723539 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:58.723928 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:58.724033 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:58.724124 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:58.724247 (+ 123us) server_negotiation.cc:299] Negotiation successful
0618 22:32:58.724330 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":295,"threads_started":1}
I20260618 22:32:58.736394 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:58.732048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:53585 (local address 127.18.150.130:39121)
0618 22:32:58.732191 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0618 22:32:58.732195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:32:58.732459 (+ 264us) server_negotiation.cc:407] Connection header received
0618 22:32:58.732583 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:32:58.732587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:32:58.732646 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:32:58.732712 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:32:58.733112 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:58.733787 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:58.735075 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:32:58.735431 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:32:58.735517 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:58.735859 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:32:58.735943 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:32:58.736000 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0618 22:32:58.736096 (+ 96us) server_negotiation.cc:299] Negotiation successful
0618 22:32:58.736164 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":40}
I20260618 22:32:58.736471 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:32:58.731899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:39121 (local address 127.18.150.130:53585)
0618 22:32:58.732384 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0618 22:32:58.732399 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:32:58.732498 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:32:58.732739 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:32:58.732742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:32:58.732750 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:32:58.732968 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:58.732975 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:58.733908 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:58.733914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:58.734962 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:32:58.734973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:32:58.735590 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:32:58.735595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:32:58.735708 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:32:58.735741 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:32:58.736015 (+ 274us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:32:58.736021 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:32:58.736075 (+ 54us) client_negotiation.cc:240] Negotiation successful
0618 22:32:58.736170 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":16,"thread_start_us":251,"threads_started":1}
I20260618 22:32:58.743427 20924 raft_consensus.cc:1261] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Refusing update from remote peer 76d5a92bac054a028eaf283943f8f500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:58.743582 21202 raft_consensus.cc:1261] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Refusing update from remote peer 76d5a92bac054a028eaf283943f8f500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:58.743968 21358 consensus_queue.cc:1035] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:58.744290 21364 consensus_queue.cc:1035] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:58.785658 20766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:32:58.785890 20766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:32:58.788614 20766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752: Sending DeleteTablet for 3 replicas of tablet 010a4028bb3a4fc0bbcdc918a32bba7b
I20260618 22:32:58.789423 21043 tablet_service.cc:1514] Processing DeleteTablet for tablet 010a4028bb3a4fc0bbcdc918a32bba7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:58 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:33984
I20260618 22:32:58.789570 20905 tablet_service.cc:1514] Processing DeleteTablet for tablet 010a4028bb3a4fc0bbcdc918a32bba7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:58 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:34816
I20260618 22:32:58.789695 21378 tablet_replica.cc:330] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500: stopping tablet replica
I20260618 22:32:58.789675 21182 tablet_service.cc:1514] Processing DeleteTablet for tablet 010a4028bb3a4fc0bbcdc918a32bba7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:32:58 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:37620
I20260618 22:32:58.789880 21377 tablet_replica.cc:330] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67: stopping tablet replica
I20260618 22:32:58.789875 21378 raft_consensus.cc:2229] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:32:58.789932 21379 tablet_replica.cc:330] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710: stopping tablet replica
I20260618 22:32:58.790114 21377 raft_consensus.cc:2229] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:32:58.790129 21379 raft_consensus.cc:2229] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:32:58.790302 21377 raft_consensus.cc:2258] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:58.790302 21379 raft_consensus.cc:2258] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:58.790355 21378 raft_consensus.cc:2258] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:32:58.790346 20766 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 22:32:58.791450 21378 ts_tablet_manager.cc:1905] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 22:32:58.791559 20766 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:32:58.791759 21377 ts_tablet_manager.cc:1905] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:58.794947 21379 ts_tablet_manager.cc:1905] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:32:58.798298 21377 ts_tablet_manager.cc:1918] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:32:58.798528 21377 log.cc:1196] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/010a4028bb3a4fc0bbcdc918a32bba7b
I20260618 22:32:58.799006 21377 ts_tablet_manager.cc:1939] T 010a4028bb3a4fc0bbcdc918a32bba7b P b5b4b5b6f36641c39c5a020413c59c67: Deleting consensus metadata
I20260618 22:32:58.800091 20753 catalog_manager.cc:4841] TS b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121): tablet 010a4028bb3a4fc0bbcdc918a32bba7b (table test-table [id=61d0f3e152c44adf81e982477565ce1c]) successfully deleted
I20260618 22:32:58.800495 21378 ts_tablet_manager.cc:1918] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:32:58.800711 21378 log.cc:1196] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/010a4028bb3a4fc0bbcdc918a32bba7b
I20260618 22:32:58.801141 21379 ts_tablet_manager.cc:1918] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:32:58.801286 21379 log.cc:1196] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/010a4028bb3a4fc0bbcdc918a32bba7b
I20260618 22:32:58.801376 21378 ts_tablet_manager.cc:1939] T 010a4028bb3a4fc0bbcdc918a32bba7b P 76d5a92bac054a028eaf283943f8f500: Deleting consensus metadata
I20260618 22:32:58.801676 21379 ts_tablet_manager.cc:1939] T 010a4028bb3a4fc0bbcdc918a32bba7b P 5d0f526169e44a129e3afce5a1194710: Deleting consensus metadata
I20260618 22:32:58.802800 20753 catalog_manager.cc:4841] TS 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943): tablet 010a4028bb3a4fc0bbcdc918a32bba7b (table test-table [id=61d0f3e152c44adf81e982477565ce1c]) successfully deleted
I20260618 22:32:58.803340 20754 catalog_manager.cc:4841] TS 76d5a92bac054a028eaf283943f8f500 (127.18.150.130:39121): tablet 010a4028bb3a4fc0bbcdc918a32bba7b (table test-table [id=61d0f3e152c44adf81e982477565ce1c]) successfully deleted
I20260618 22:32:58.804641 20905 tablet_service.cc:1467] Processing CreateTablet for tablet c01d9bd1727c4bd195eec9577d4ca038 (DEFAULT_TABLE table=test-table [id=2fbcfc63340d4aa48a6241bc625651c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:58.804852 21043 tablet_service.cc:1467] Processing CreateTablet for tablet c01d9bd1727c4bd195eec9577d4ca038 (DEFAULT_TABLE table=test-table [id=2fbcfc63340d4aa48a6241bc625651c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:58.805110 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c01d9bd1727c4bd195eec9577d4ca038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:58.805204 21043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c01d9bd1727c4bd195eec9577d4ca038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:58.805279 21182 tablet_service.cc:1467] Processing CreateTablet for tablet c01d9bd1727c4bd195eec9577d4ca038 (DEFAULT_TABLE table=test-table [id=2fbcfc63340d4aa48a6241bc625651c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:32:58.805639 21182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c01d9bd1727c4bd195eec9577d4ca038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:32:58.809247 21389 tablet_bootstrap.cc:492] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500: Bootstrap starting.
I20260618 22:32:58.810950 21389 tablet_bootstrap.cc:654] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:58.811305 21390 tablet_bootstrap.cc:492] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67: Bootstrap starting.
I20260618 22:32:58.811444 21388 tablet_bootstrap.cc:492] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710: Bootstrap starting.
I20260618 22:32:58.812839 21390 tablet_bootstrap.cc:654] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:58.812903 21388 tablet_bootstrap.cc:654] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710: Neither blocks nor log segments found. Creating new log.
I20260618 22:32:58.822642 21390 tablet_bootstrap.cc:492] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67: No bootstrap required, opened a new log
I20260618 22:32:58.822876 21388 tablet_bootstrap.cc:492] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710: No bootstrap required, opened a new log
I20260618 22:32:58.822960 21389 tablet_bootstrap.cc:492] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500: No bootstrap required, opened a new log
I20260618 22:32:58.822898 21390 ts_tablet_manager.cc:1397] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260618 22:32:58.823040 21388 ts_tablet_manager.cc:1397] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260618 22:32:58.823119 21389 ts_tablet_manager.cc:1397] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260618 22:32:58.823832 21390 raft_consensus.cc:348] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.823843 21389 raft_consensus.cc:348] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.824098 21389 raft_consensus.cc:374] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:58.824201 21389 raft_consensus.cc:729] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d5a92bac054a028eaf283943f8f500, State: Initialized, Role: FOLLOWER
I20260618 22:32:58.824231 21390 raft_consensus.cc:374] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:58.824390 21390 raft_consensus.cc:729] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b4b5b6f36641c39c5a020413c59c67, State: Initialized, Role: FOLLOWER
I20260618 22:32:58.824456 21389 consensus_queue.cc:260] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.824826 21390 consensus_queue.cc:260] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.824971 21389 ts_tablet_manager.cc:1428] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:32:58.824934 21388 raft_consensus.cc:348] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.825186 21388 raft_consensus.cc:374] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:32:58.825255 21388 raft_consensus.cc:729] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d0f526169e44a129e3afce5a1194710, State: Initialized, Role: FOLLOWER
I20260618 22:32:58.825543 21388 consensus_queue.cc:260] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:58.827109 21388 ts_tablet_manager.cc:1428] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260618 22:32:58.827689 21390 ts_tablet_manager.cc:1428] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260618 22:32:59.159994 21364 raft_consensus.cc:482] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:32:59.160193 21364 raft_consensus.cc:504] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:59.160789 21364 leader_election.cc:288] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121), 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943)
I20260618 22:32:59.161353 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01d9bd1727c4bd195eec9577d4ca038" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b4b5b6f36641c39c5a020413c59c67" is_pre_election: true
I20260618 22:32:59.161429 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01d9bd1727c4bd195eec9577d4ca038" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d0f526169e44a129e3afce5a1194710" is_pre_election: true
I20260618 22:32:59.161590 20925 raft_consensus.cc:2454] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 0.
I20260618 22:32:59.161696 21203 raft_consensus.cc:2454] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 0.
I20260618 22:32:59.162101 20996 leader_election.cc:302] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76d5a92bac054a028eaf283943f8f500, b5b4b5b6f36641c39c5a020413c59c67; no voters:
I20260618 22:32:59.162360 21364 raft_consensus.cc:2781] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:32:59.162484 21364 raft_consensus.cc:482] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:32:59.162570 21364 raft_consensus.cc:3037] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:59.164666 21364 raft_consensus.cc:504] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:59.165170 21364 leader_election.cc:288] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 election: Requested vote from peers b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121), 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943)
I20260618 22:32:59.165717 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01d9bd1727c4bd195eec9577d4ca038" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b4b5b6f36641c39c5a020413c59c67"
I20260618 22:32:59.165758 21203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01d9bd1727c4bd195eec9577d4ca038" candidate_uuid: "76d5a92bac054a028eaf283943f8f500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d0f526169e44a129e3afce5a1194710"
I20260618 22:32:59.165956 21203 raft_consensus.cc:3037] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:59.165984 20925 raft_consensus.cc:3037] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:32:59.167950 21203 raft_consensus.cc:2454] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 1.
I20260618 22:32:59.168511 20996 leader_election.cc:302] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d0f526169e44a129e3afce5a1194710, 76d5a92bac054a028eaf283943f8f500; no voters:
I20260618 22:32:59.168808 21358 raft_consensus.cc:2781] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:32:59.169000 21358 raft_consensus.cc:686] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 1 LEADER]: Becoming Leader. State: Replica: 76d5a92bac054a028eaf283943f8f500, State: Running, Role: LEADER
I20260618 22:32:59.169255 21358 consensus_queue.cc:237] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } }
I20260618 22:32:59.169878 20925 raft_consensus.cc:2454] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76d5a92bac054a028eaf283943f8f500 in term 1.
I20260618 22:32:59.172199 20767 catalog_manager.cc:5495] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76d5a92bac054a028eaf283943f8f500 (127.18.150.130). New cstate: current_term: 1 leader_uuid: "76d5a92bac054a028eaf283943f8f500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76d5a92bac054a028eaf283943f8f500" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 } health_report { overall_health: UNKNOWN } } }
I20260618 22:32:59.198555 21203 raft_consensus.cc:1261] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Refusing update from remote peer 76d5a92bac054a028eaf283943f8f500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:59.198655 20924 raft_consensus.cc:1261] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Refusing update from remote peer 76d5a92bac054a028eaf283943f8f500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:32:59.199052 21334 consensus_queue.cc:1035] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d0f526169e44a129e3afce5a1194710" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:32:59.199389 21408 consensus_queue.cc:1035] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b4b5b6f36641c39c5a020413c59c67" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:32:59.216962 21356 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260618 22:33:00.791111 20767 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:00.791381 20767 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:00.795531 20767 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffae639e22b24681b9074ef9d509f752: Sending DeleteTablet for 3 replicas of tablet c01d9bd1727c4bd195eec9577d4ca038
I20260618 22:33:00.796468 21044 tablet_service.cc:1514] Processing DeleteTablet for tablet c01d9bd1727c4bd195eec9577d4ca038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:00 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:33984
I20260618 22:33:00.796487 20905 tablet_service.cc:1514] Processing DeleteTablet for tablet c01d9bd1727c4bd195eec9577d4ca038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:00 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:34816
I20260618 22:33:00.796926 21182 tablet_service.cc:1514] Processing DeleteTablet for tablet c01d9bd1727c4bd195eec9577d4ca038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:00 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:37620
I20260618 22:33:00.797089 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20839
I20260618 22:33:00.797164 21431 tablet_replica.cc:330] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67: stopping tablet replica
I20260618 22:33:00.797361 21431 raft_consensus.cc:2229] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:00.797595 21431 raft_consensus.cc:2258] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:00.798717 21431 ts_tablet_manager.cc:1905] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:00.806594 21431 ts_tablet_manager.cc:1918] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:33:00.806844 21431 log.cc:1196] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/c01d9bd1727c4bd195eec9577d4ca038
I20260618 22:33:00.807338 21431 ts_tablet_manager.cc:1939] T c01d9bd1727c4bd195eec9577d4ca038 P b5b4b5b6f36641c39c5a020413c59c67: Deleting consensus metadata
I20260618 22:33:00.808725 20753 catalog_manager.cc:4841] TS b5b4b5b6f36641c39c5a020413c59c67 (127.18.150.129:41121): tablet c01d9bd1727c4bd195eec9577d4ca038 (table test-table [id=2fbcfc63340d4aa48a6241bc625651c4]) successfully deleted
I20260618 22:33:00.812583 21430 tablet_replica.cc:330] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500: stopping tablet replica
I20260618 22:33:00.812822 21430 raft_consensus.cc:2229] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:33:00.813230 21430 raft_consensus.cc:2258] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:00.814570 21430 ts_tablet_manager.cc:1905] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:00.814767 21432 tablet_replica.cc:330] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710: stopping tablet replica
I20260618 22:33:00.814954 21432 raft_consensus.cc:2229] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:00.815263 21432 raft_consensus.cc:2258] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:00.816201 21432 ts_tablet_manager.cc:1905] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:00.821015 21430 ts_tablet_manager.cc:1918] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:33:00.821161 21430 log.cc:1196] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/c01d9bd1727c4bd195eec9577d4ca038
I20260618 22:33:00.821635 21430 ts_tablet_manager.cc:1939] T c01d9bd1727c4bd195eec9577d4ca038 P 76d5a92bac054a028eaf283943f8f500: Deleting consensus metadata
Jun 18 22:33:00 dist-test-slave-h7jz krb5kdc[20723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821978, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:33:00.822572 21432 ts_tablet_manager.cc:1918] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:33:00.822705 21432 log.cc:1196] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/c01d9bd1727c4bd195eec9577d4ca038
I20260618 22:33:00.822757 20754 catalog_manager.cc:4841] TS 76d5a92bac054a028eaf283943f8f500 (127.18.150.130:39121): tablet c01d9bd1727c4bd195eec9577d4ca038 (table test-table [id=2fbcfc63340d4aa48a6241bc625651c4]) successfully deleted
I20260618 22:33:00.823094 21432 ts_tablet_manager.cc:1939] T c01d9bd1727c4bd195eec9577d4ca038 P 5d0f526169e44a129e3afce5a1194710: Deleting consensus metadata
I20260618 22:33:00.824007 20753 catalog_manager.cc:4841] TS 5d0f526169e44a129e3afce5a1194710 (127.18.150.131:42943): tablet c01d9bd1727c4bd195eec9577d4ca038 (table test-table [id=2fbcfc63340d4aa48a6241bc625651c4]) successfully deleted
I20260618 22:33:00.829025 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:00.799692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52520 (local address 127.18.150.129:41121)
0618 22:33:00.813827 (+ 14135us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:00.813835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:00.814632 (+ 797us) server_negotiation.cc:407] Connection header received
0618 22:33:00.814799 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:00.814805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:00.814889 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:00.814998 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:00.816435 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:00.817237 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:00.818630 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:00.819006 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:00.819112 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:00.823521 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:00.823537 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:00.823555 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:00.823588 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:00.826428 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:00.827081 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:00.827089 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:00.827091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:00.827230 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:00.827587 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:00.827591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:00.827593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:00.827928 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:00.828224 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:00.828489 (+ 265us) server_negotiation.cc:299] Negotiation successful
0618 22:33:00.828698 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13972,"thread_start_us":13855,"threads_started":1}
W20260618 22:33:00.829674 20970 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52520
W20260618 22:33:00.830293 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20839: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:33:00.830511 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 20839
I20260618 22:33:00.847985 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20979
Jun 18 22:33:00 dist-test-slave-h7jz krb5kdc[20723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821978, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:33:00.861550 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:00.848624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42700 (local address 127.18.150.130:39121)
0618 22:33:00.849088 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:00.849093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:00.849109 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:33:00.849160 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:00.849163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:00.849222 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:00.849307 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:00.850519 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:00.851262 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:00.852458 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:00.852899 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:00.853001 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:00.856743 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:00.856757 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:00.856773 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:00.856806 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:00.859401 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:00.859994 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:00.860001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:00.860003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:00.860144 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:00.860407 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:00.860410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:00.860412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:00.860738 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:00.860984 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:00.861135 (+ 151us) server_negotiation.cc:299] Negotiation successful
0618 22:33:00.861292 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":246,"threads_started":1}
W20260618 22:33:00.862257 21109 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42700
W20260618 22:33:00.862883 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20979: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:33:00.863049 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 20979
I20260618 22:33:00.882711 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 21118
Jun 18 22:33:00 dist-test-slave-h7jz krb5kdc[20723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821978, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:33:00.896704 21436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:00.883395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45802 (local address 127.18.150.131:42943)
0618 22:33:00.883880 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:00.883884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:00.883901 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:33:00.883955 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:00.883958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:00.884023 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:00.884110 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:00.885308 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:00.886112 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:00.887298 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:00.887661 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:00.887750 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:00.891711 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:00.891725 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:00.891741 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:00.891772 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:00.894347 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:00.894927 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:00.894944 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:00.894947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:00.895085 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:00.895407 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:00.895411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:00.895413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:00.895805 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:00.896058 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:00.896235 (+ 177us) server_negotiation.cc:299] Negotiation successful
0618 22:33:00.896394 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":279,"threads_started":1}
W20260618 22:33:00.897305 21244 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45802
W20260618 22:33:00.897925 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 21118: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:33:00.898161 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 21118
I20260618 22:33:00.914728 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 20736
I20260618 22:33:00.915931 20808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:01.027462 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 20736
2026-06-18T22:33:01Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8540 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:33:01 dist-test-slave-h7jz krb5kdc[21440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:33:01 dist-test-slave-h7jz krb5kdc[21440](info): set up 2 sockets
Jun 18 22:33:01 dist-test-slave-h7jz krb5kdc[21440](info): commencing operation
krb5kdc: starting...
W20260618 22:33:02.013825 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:33:02 dist-test-slave-h7jz krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:33:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:33:02.189980 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35819
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:35819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:02.502048 21453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:02.502346 21453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:02.502475 21453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:02.509614 21453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:02.509721 21453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:02.509783 21453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:02.509856 21453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:02.509902 21453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:02.519724 21453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:35819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:02.521157 21453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:02.522575 21453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:02.533627 21458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:02.533690 21459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:02.533667 21461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:02.534449 21453 server_base.cc:1034] running on GCE node
I20260618 22:33:02.535197 21453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:02.536567 21453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:02.537784 21453 hybrid_clock.cc:648] HybridClock initialized: now 1781821982537727 us; error 48 us; skew 500 ppm
I20260618 22:33:02.538161 21453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:02 dist-test-slave-h7jz krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821982, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:02.541208 21453 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:33:02.542742 21453 webserver.cc:460] Webserver started at http://127.18.150.190:33013/ using document root <none> and password file <none>
I20260618 22:33:02.543431 21453 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:02.543545 21453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:02.543834 21453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:02.546840 21453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "e591a38bdc8441dc8356ea8982342986"
format_stamp: "Formatted at 2026-06-18 22:33:02 on dist-test-slave-h7jz"
server_key: "f6fea72f5fcc570b439cb43153fd1bdb"
server_key_iv: "47d0d488abe36c2337ae489db9d65d47"
server_key_version: "encryptionkey@0"
I20260618 22:33:02.547549 21453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "e591a38bdc8441dc8356ea8982342986"
format_stamp: "Formatted at 2026-06-18 22:33:02 on dist-test-slave-h7jz"
server_key: "f6fea72f5fcc570b439cb43153fd1bdb"
server_key_iv: "47d0d488abe36c2337ae489db9d65d47"
server_key_version: "encryptionkey@0"
I20260618 22:33:02.552567 21453 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:33:02.556041 21468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:02.557410 21453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:33:02.557578 21453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "e591a38bdc8441dc8356ea8982342986"
format_stamp: "Formatted at 2026-06-18 22:33:02 on dist-test-slave-h7jz"
server_key: "f6fea72f5fcc570b439cb43153fd1bdb"
server_key_iv: "47d0d488abe36c2337ae489db9d65d47"
server_key_version: "encryptionkey@0"
I20260618 22:33:02.557766 21453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:02.573668 21453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:02.577816 21453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:02.578097 21453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:02.595147 21453 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:35819
I20260618 22:33:02.595180 21519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:35819 every 8 connection(s)
I20260618 22:33:02.596483 21453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:02.600106 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:02.606598 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 21453
I20260618 22:33:02.606835 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:02.607560 19034 external_mini_cluster.cc:1428] Setting key dcd48d0575e67d2169b69e1b79d731f1
I20260618 22:33:02.610633 21520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986: Bootstrap starting.
I20260618 22:33:02.614569 21520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:02.615612 21520 log.cc:824] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:02.618338 21520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986: No bootstrap required, opened a new log
Jun 18 22:33:02 dist-test-slave-h7jz krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:02.625558 21520 raft_consensus.cc:348] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591a38bdc8441dc8356ea8982342986" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35819 } }
I20260618 22:33:02.625885 21520 raft_consensus.cc:374] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:02.626024 21520 raft_consensus.cc:729] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e591a38bdc8441dc8356ea8982342986, State: Initialized, Role: FOLLOWER
I20260618 22:33:02.626609 21520 consensus_queue.cc:260] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591a38bdc8441dc8356ea8982342986" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35819 } }
I20260618 22:33:02.626802 21520 raft_consensus.cc:388] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:02.626933 21520 raft_consensus.cc:482] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:02.627113 21520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:02.629281 21520 raft_consensus.cc:504] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591a38bdc8441dc8356ea8982342986" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35819 } }
I20260618 22:33:02.629732 21520 leader_election.cc:302] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e591a38bdc8441dc8356ea8982342986; no voters:
I20260618 22:33:02.630286 21523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:02.609812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41954 (local address 127.18.150.190:35819)
0618 22:33:02.610768 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:02.610784 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:02.611142 (+ 358us) server_negotiation.cc:407] Connection header received
0618 22:33:02.613578 (+ 2436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:02.613607 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:02.614113 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:02.614550 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:02.615840 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:02.617426 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:02.618699 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:02.619354 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:02.619512 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:02.623307 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:02.623315 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:02.623331 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:02.623381 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:02.626349 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:02.626972 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:02.626981 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:02.626989 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:02.627179 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:02.627519 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:02.627527 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:02.627531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:02.628063 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:02.628337 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:02.628780 (+ 443us) server_negotiation.cc:299] Negotiation successful
0618 22:33:02.629164 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":396,"threads_started":1}
I20260618 22:33:02.630357 21520 leader_election.cc:288] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:02.630476 21525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:02.632543 21520 sys_catalog.cc:564] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:02.634239 21525 raft_consensus.cc:686] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [term 1 LEADER]: Becoming Leader. State: Replica: e591a38bdc8441dc8356ea8982342986, State: Running, Role: LEADER
I20260618 22:33:02.634864 21525 consensus_queue.cc:237] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591a38bdc8441dc8356ea8982342986" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35819 } }
I20260618 22:33:02.637991 21526 sys_catalog.cc:455] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e591a38bdc8441dc8356ea8982342986" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591a38bdc8441dc8356ea8982342986" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35819 } } }
I20260618 22:33:02.638348 21526 sys_catalog.cc:458] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:02.639808 21533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:02.639725 21527 sys_catalog.cc:455] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e591a38bdc8441dc8356ea8982342986. Latest consensus state: current_term: 1 leader_uuid: "e591a38bdc8441dc8356ea8982342986" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591a38bdc8441dc8356ea8982342986" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35819 } } }
I20260618 22:33:02.640002 21527 sys_catalog.cc:458] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:02.644678 21533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:02.654444 21533 catalog_manager.cc:1347] Generated new cluster ID: 4d6dff38a9384349a4563143c151dbe2
I20260618 22:33:02.654549 21533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:02.670495 21533 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:02.671927 21533 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:02.681740 21533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e591a38bdc8441dc8356ea8982342986: Generated new TSK 0
I20260618 22:33:02.682883 21533 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:33:02.729475 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35819
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:03.043697 21546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:03.043972 21546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:03.044088 21546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:03.051488 21546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:03.051613 21546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:03.051764 21546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:33:03.062153 21546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:03.063488 21546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:03.064852 21546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:03.077317 21554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:03.077383 21552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:03.077564 21551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:03.078336 21546 server_base.cc:1034] running on GCE node
I20260618 22:33:03.078912 21546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:03.079769 21546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:03.080955 21546 hybrid_clock.cc:648] HybridClock initialized: now 1781821983080906 us; error 44 us; skew 500 ppm
I20260618 22:33:03.081243 21546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:03 dist-test-slave-h7jz krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821983, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:03.084329 21546 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:33:03.085837 21546 webserver.cc:460] Webserver started at http://127.18.150.129:39199/ using document root <none> and password file <none>
I20260618 22:33:03.086565 21546 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:03.086678 21546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:03.086966 21546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:03.089819 21546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "4e2de60b456a448c93f6f70ab079bfaa"
format_stamp: "Formatted at 2026-06-18 22:33:03 on dist-test-slave-h7jz"
server_key: "06e4012d9ab7fce742ee07b0b8983d37"
server_key_iv: "45817f6b6151da7eaf36be2867af36ea"
server_key_version: "encryptionkey@0"
I20260618 22:33:03.090607 21546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "4e2de60b456a448c93f6f70ab079bfaa"
format_stamp: "Formatted at 2026-06-18 22:33:03 on dist-test-slave-h7jz"
server_key: "06e4012d9ab7fce742ee07b0b8983d37"
server_key_iv: "45817f6b6151da7eaf36be2867af36ea"
server_key_version: "encryptionkey@0"
I20260618 22:33:03.095733 21546 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:33:03.099059 21561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:03.100364 21546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:33:03.100503 21546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "4e2de60b456a448c93f6f70ab079bfaa"
format_stamp: "Formatted at 2026-06-18 22:33:03 on dist-test-slave-h7jz"
server_key: "06e4012d9ab7fce742ee07b0b8983d37"
server_key_iv: "45817f6b6151da7eaf36be2867af36ea"
server_key_version: "encryptionkey@0"
I20260618 22:33:03.100684 21546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:03.116840 21546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:03.120797 21546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:03.121043 21546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:03.121930 21546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:03.123227 21546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:03.123294 21546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:03.123373 21546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:03.123414 21546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:03.152045 21546 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:38581
I20260618 22:33:03.152112 21673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:38581 every 8 connection(s)
I20260618 22:33:03.153254 21546 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:33:03.157465 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 21546
I20260618 22:33:03.157648 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:33:03.158239 19034 external_mini_cluster.cc:1428] Setting key 2cce2b07b09dd6cd68c42d9a92b2171d
Jun 18 22:33:03 dist-test-slave-h7jz krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821983, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:03.172379 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:03.155650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:49185 (local address 127.18.150.190:35819)
0618 22:33:03.156032 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:03.156036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:03.156933 (+ 897us) server_negotiation.cc:407] Connection header received
0618 22:33:03.158839 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:03.158844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:03.158915 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:03.159028 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:03.160735 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:03.161469 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:03.163060 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:03.163442 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:03.163546 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:03.167009 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:03.167014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:03.167017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:03.167048 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:03.169414 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:03.170207 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:03.170211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:03.170214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:03.170330 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:03.170740 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:03.170744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:03.170746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:03.171002 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:03.171129 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:03.171813 (+ 684us) server_negotiation.cc:299] Negotiation successful
0618 22:33:03.172031 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":192,"threads_started":1}
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
I20260618 22:33:03.173465 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:03.156100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35819 (local address 127.18.150.129:49185)
0618 22:33:03.156780 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:03.156822 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:03.158554 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:03.159257 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:03.159268 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:03.159946 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:03.160578 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:03.160595 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:03.161604 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:03.161608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:03.162932 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:03.162943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:03.163639 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:03.163645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:03.163861 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:03.164461 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:03.164483 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:03.166804 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:03.169561 (+ 2757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:03.169571 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:03.169590 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:03.170078 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:03.170434 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:03.170439 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:03.170443 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:03.170633 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:03.171117 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:03.171135 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:03.171472 (+ 337us) client_negotiation.cc:769] Sending connection context
0618 22:33:03.171708 (+ 236us) client_negotiation.cc:240] Negotiation successful
0618 22:33:03.172321 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":253,"threads_started":1}
I20260618 22:33:03.175096 21674 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35819
I20260618 22:33:03.175422 21674 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
I20260618 22:33:03.176223 21674 heartbeater.cc:511] Master 127.18.150.190:35819 requested a full tablet report, sending...
I20260618 22:33:03.178294 21485 ts_manager.cc:194] Registered new tserver with Master: 4e2de60b456a448c93f6f70ab079bfaa (127.18.150.129:38581)
I20260618 22:33:03.179731 21485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:49185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:33:03.196962 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35819
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:03.503183 21680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:03.503473 21680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:03.503602 21680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:03.510869 21680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:03.510989 21680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:03.511137 21680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:33:03.520999 21680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:03.522353 21680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:03.523715 21680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:03.536337 21685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:03.536405 21688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:03.536424 21686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:03.537418 21680 server_base.cc:1034] running on GCE node
I20260618 22:33:03.538077 21680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:03.539024 21680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:03.540215 21680 hybrid_clock.cc:648] HybridClock initialized: now 1781821983540157 us; error 51 us; skew 500 ppm
I20260618 22:33:03.540534 21680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:03 dist-test-slave-h7jz krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821983, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:03.543581 21680 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:33:03.545040 21680 webserver.cc:460] Webserver started at http://127.18.150.130:36505/ using document root <none> and password file <none>
I20260618 22:33:03.545732 21680 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:03.545842 21680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:03.546160 21680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:03.549008 21680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "ce765e6c52da407cbe36d181210bf6f4"
format_stamp: "Formatted at 2026-06-18 22:33:03 on dist-test-slave-h7jz"
server_key: "2ab9e5f51ae2950e2e4c261823adab45"
server_key_iv: "ed78be63444f1ff78d374fbfa27927de"
server_key_version: "encryptionkey@0"
I20260618 22:33:03.549755 21680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "ce765e6c52da407cbe36d181210bf6f4"
format_stamp: "Formatted at 2026-06-18 22:33:03 on dist-test-slave-h7jz"
server_key: "2ab9e5f51ae2950e2e4c261823adab45"
server_key_iv: "ed78be63444f1ff78d374fbfa27927de"
server_key_version: "encryptionkey@0"
I20260618 22:33:03.554834 21680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260618 22:33:03.558120 21695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:03.559458 21680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:33:03.559628 21680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "ce765e6c52da407cbe36d181210bf6f4"
format_stamp: "Formatted at 2026-06-18 22:33:03 on dist-test-slave-h7jz"
server_key: "2ab9e5f51ae2950e2e4c261823adab45"
server_key_iv: "ed78be63444f1ff78d374fbfa27927de"
server_key_version: "encryptionkey@0"
I20260618 22:33:03.559803 21680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:03.599792 21680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:03.605288 21680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:03.605585 21680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:03.606755 21680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:03.608443 21680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:03.608538 21680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:03.608656 21680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:03.608711 21680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:03.638816 21680 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:46621
I20260618 22:33:03.638870 21807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:46621 every 8 connection(s)
I20260618 22:33:03.640050 21680 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:33:03.644001 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 21680
I20260618 22:33:03.644214 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:33:03.644706 19034 external_mini_cluster.cc:1428] Setting key 0093cfdf30c8bf2404660c320987816f
Jun 18 22:33:03 dist-test-slave-h7jz krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821983, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:03.661077 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:03.642503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:36935 (local address 127.18.150.190:35819)
0618 22:33:03.642688 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:03.642692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:03.643883 (+ 1191us) server_negotiation.cc:407] Connection header received
0618 22:33:03.646108 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:03.646114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:03.646186 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:03.646292 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:03.648161 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:03.648849 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:03.650954 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:03.651349 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:03.651444 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:03.654956 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:03.654961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:03.654965 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:03.655002 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:03.657997 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:03.658800 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:03.658805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:03.658809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:03.658965 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:03.659346 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:03.659351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:03.659355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:03.659694 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:03.659841 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:03.660502 (+ 661us) server_negotiation.cc:299] Negotiation successful
0618 22:33:03.660731 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:33:03.661998 21810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:03.643009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35819 (local address 127.18.150.130:36935)
0618 22:33:03.643691 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:03.643741 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:03.645837 (+ 2096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:03.646521 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:03.646530 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:03.647221 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:03.647983 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:03.648008 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:03.648985 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:03.648990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:03.650835 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:03.650848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:03.651555 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:03.651561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:03.651865 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:03.652525 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:03.652546 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:03.654728 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:03.658129 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:03.658138 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:03.658154 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:03.658666 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:03.659057 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:03.659060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:03.659062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:03.659242 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:03.659799 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:03.659804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:03.660126 (+ 322us) client_negotiation.cc:769] Sending connection context
0618 22:33:03.660367 (+ 241us) client_negotiation.cc:240] Negotiation successful
0618 22:33:03.660801 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":239,"threads_started":1}
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:33:03.663455 21808 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35819
I20260618 22:33:03.663748 21808 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:03.664566 21808 heartbeater.cc:511] Master 127.18.150.190:35819 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
I20260618 22:33:03.666035 21485 ts_manager.cc:194] Registered new tserver with Master: ce765e6c52da407cbe36d181210bf6f4 (127.18.150.130:46621)
I20260618 22:33:03.666811 21485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:36935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:33:03.688988 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35819
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:04.004347 21813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:04.004642 21813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:04.004760 21813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:04.012077 21813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:04.012207 21813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:04.012358 21813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:33:04.022230 21813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:04.023518 21813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:04.024806 21813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:04.037230 21818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:04.037261 21821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:04.037293 21819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:04.038230 21813 server_base.cc:1034] running on GCE node
I20260618 22:33:04.038774 21813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:04.039717 21813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:04.040939 21813 hybrid_clock.cc:648] HybridClock initialized: now 1781821984040878 us; error 53 us; skew 500 ppm
I20260618 22:33:04.041285 21813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:04 dist-test-slave-h7jz krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821984, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:04.044353 21813 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:33:04.045763 21813 webserver.cc:460] Webserver started at http://127.18.150.131:34875/ using document root <none> and password file <none>
I20260618 22:33:04.046458 21813 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:04.046555 21813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:04.046803 21813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:04.049582 21813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "7faef6ac247344e782ddaa9f6f667dac"
format_stamp: "Formatted at 2026-06-18 22:33:04 on dist-test-slave-h7jz"
server_key: "dbf5a215339267e0b20f22a18e882cfb"
server_key_iv: "64f6769bae08dfb39defc08fdcc1eb81"
server_key_version: "encryptionkey@0"
I20260618 22:33:04.050338 21813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "7faef6ac247344e782ddaa9f6f667dac"
format_stamp: "Formatted at 2026-06-18 22:33:04 on dist-test-slave-h7jz"
server_key: "dbf5a215339267e0b20f22a18e882cfb"
server_key_iv: "64f6769bae08dfb39defc08fdcc1eb81"
server_key_version: "encryptionkey@0"
I20260618 22:33:04.055260 21813 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260618 22:33:04.058631 21828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:04.059971 21813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:33:04.060138 21813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "7faef6ac247344e782ddaa9f6f667dac"
format_stamp: "Formatted at 2026-06-18 22:33:04 on dist-test-slave-h7jz"
server_key: "dbf5a215339267e0b20f22a18e882cfb"
server_key_iv: "64f6769bae08dfb39defc08fdcc1eb81"
server_key_version: "encryptionkey@0"
I20260618 22:33:04.060317 21813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:04.081212 21813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:04.085237 21813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:04.085496 21813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:04.086462 21813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:04.087742 21813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:04.087821 21813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:04.087944 21813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:04.087994 21813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:04.117249 21813 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:33159
I20260618 22:33:04.117291 21940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:33159 every 8 connection(s)
I20260618 22:33:04.118516 21813 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:33:04.125787 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 21813
I20260618 22:33:04.125996 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:33:04.126379 19034 external_mini_cluster.cc:1428] Setting key f1df883f19b84dca9825088ba4a206d1
Jun 18 22:33:04 dist-test-slave-h7jz krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821984, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:04.137758 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:04.120921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:36005 (local address 127.18.150.190:35819)
0618 22:33:04.121095 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:04.121100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:04.122279 (+ 1179us) server_negotiation.cc:407] Connection header received
0618 22:33:04.124061 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:04.124066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:04.124146 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:04.124245 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:04.125848 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:04.126668 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:04.128463 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:04.128926 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:04.129050 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:04.132362 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:04.132366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:04.132369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:04.132399 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:04.134885 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:04.135593 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:04.135597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:04.135600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:04.135720 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:04.136112 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:04.136117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:04.136120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:04.136387 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:04.136512 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:04.137177 (+ 665us) server_negotiation.cc:299] Negotiation successful
0618 22:33:04.137395 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:33:04.138860 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:04.121427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35819 (local address 127.18.150.131:36005)
0618 22:33:04.122137 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:04.122175 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:04.123757 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:04.124487 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:04.124495 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:04.124988 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:04.125675 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:04.125695 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:04.126817 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:04.126820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:04.128341 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:04.128352 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:04.129150 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:04.129159 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:04.129384 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:04.129990 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:04.130015 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:04.132164 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:04.135030 (+ 2866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:04.135040 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:04.135058 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:04.135485 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:04.135819 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:04.135825 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:04.135829 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:04.136014 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:04.136504 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:04.136513 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:04.136846 (+ 333us) client_negotiation.cc:769] Sending connection context
0618 22:33:04.137087 (+ 241us) client_negotiation.cc:240] Negotiation successful
0618 22:33:04.137675 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":249,"threads_started":1}
I20260618 22:33:04.140345 21941 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35819
I20260618 22:33:04.140661 21941 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:04.141429 21941 heartbeater.cc:511] Master 127.18.150.190:35819 requested a full tablet report, sending...
I20260618 22:33:04.142814 21485 ts_manager.cc:194] Registered new tserver with Master: 7faef6ac247344e782ddaa9f6f667dac (127.18.150.131:33159)
I20260618 22:33:04.143591 21485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:36005
I20260618 22:33:04.153914 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:33:04.182806 21674 heartbeater.cc:503] Master 127.18.150.190:35819 was elected leader, sending a full tablet report...
W20260618 22:33:04.183965 21676 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:04.181520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.18.150.190:35819)
0618 22:33:04.181753 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:04.181759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:04.182134 (+ 375us) server_negotiation.cc:407] Connection header received
0618 22:33:04.182326 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:04.182331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:04.182404 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:04.182497 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:04.183711 (+ 1214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41966: BlockingRecv error: recv got EOF from 127.0.0.1:41966 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:33:04.186949 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 21546
W20260618 22:33:04.189558 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 21546: Not authorized: Client connection negotiation failed: client connection to 127.18.150.129:38581: server requires authentication, but client does not have Kerberos credentials available
I20260618 22:33:04.189736 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 21546
I20260618 22:33:04.201615 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 21680
W20260618 22:33:04.204114 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 21680: Not authorized: Client connection negotiation failed: client connection to 127.18.150.130:46621: server requires authentication, but client does not have Kerberos credentials available
I20260618 22:33:04.204262 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 21680
I20260618 22:33:04.215947 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 21813
W20260618 22:33:04.218657 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 21813: Not authorized: Client connection negotiation failed: client connection to 127.18.150.131:33159: server requires authentication, but client does not have Kerberos credentials available
I20260618 22:33:04.218801 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 21813
I20260618 22:33:04.230358 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 21453
I20260618 22:33:04.231501 21515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:04.332531 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 21453
2026-06-18T22:33:04Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3301 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:33:04 dist-test-slave-h7jz krb5kdc[21958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:33:04 dist-test-slave-h7jz krb5kdc[21958](info): set up 2 sockets
Jun 18 22:33:04 dist-test-slave-h7jz krb5kdc[21958](info): commencing operation
krb5kdc: starting...
W20260618 22:33:05.325925 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:33:05 dist-test-slave-h7jz krb5kdc[21958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:33:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:33:05.501853 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:40179
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:41561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:40179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:05.814951 21971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:05.815256 21971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:05.815389 21971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:05.825254 21971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:05.825402 21971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:05.825464 21971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:05.825579 21971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:05.825650 21971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:05.839262 21971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:41561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:40179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:40179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:05.840932 21971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:05.842691 21971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:05.856315 21979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:05.856717 21977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:05.857033 21976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:05.858057 21971 server_base.cc:1034] running on GCE node
I20260618 22:33:05.858817 21971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:05.862058 21971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:05.863286 21971 hybrid_clock.cc:648] HybridClock initialized: now 1781821985863218 us; error 54 us; skew 500 ppm
I20260618 22:33:05.863654 21971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:05 dist-test-slave-h7jz krb5kdc[21958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:05.866724 21971 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:33:05.868382 21971 webserver.cc:460] Webserver started at http://127.18.150.190:44507/ using document root <none> and password file <none>
I20260618 22:33:05.869060 21971 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:05.869164 21971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:05.869419 21971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:05.872355 21971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "e00d5c4fff4e492ca92c800c4064a4ee"
format_stamp: "Formatted at 2026-06-18 22:33:05 on dist-test-slave-h7jz"
server_key: "b3b8f0545d513e3a4467cf442511ad52"
server_key_iv: "a2dedc18ad90b41f777c59db823d9902"
server_key_version: "encryptionkey@0"
I20260618 22:33:05.873108 21971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "e00d5c4fff4e492ca92c800c4064a4ee"
format_stamp: "Formatted at 2026-06-18 22:33:05 on dist-test-slave-h7jz"
server_key: "b3b8f0545d513e3a4467cf442511ad52"
server_key_iv: "a2dedc18ad90b41f777c59db823d9902"
server_key_version: "encryptionkey@0"
I20260618 22:33:05.878283 21971 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260618 22:33:05.881599 21986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:05.882948 21971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:05.883097 21971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "e00d5c4fff4e492ca92c800c4064a4ee"
format_stamp: "Formatted at 2026-06-18 22:33:05 on dist-test-slave-h7jz"
server_key: "b3b8f0545d513e3a4467cf442511ad52"
server_key_iv: "a2dedc18ad90b41f777c59db823d9902"
server_key_version: "encryptionkey@0"
I20260618 22:33:05.883287 21971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:05.902010 21971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:05.906246 21971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:05.906500 21971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:05.923190 21971 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:40179
I20260618 22:33:05.923235 22037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:40179 every 8 connection(s)
I20260618 22:33:05.924530 21971 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:05.928175 22038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:05.928987 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 21971
I20260618 22:33:05.929229 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:05.930014 19034 external_mini_cluster.cc:1428] Setting key 9992da7e777b14106e4de56e0f3b8778
I20260618 22:33:05.942123 22038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee: Bootstrap starting.
Jun 18 22:33:05 dist-test-slave-h7jz krb5kdc[21958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:05.945907 22038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:05.946997 22038 log.cc:824] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:05.950183 22038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee: No bootstrap required, opened a new log
I20260618 22:33:05.952951 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:05.931830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49778 (local address 127.18.150.190:40179)
0618 22:33:05.932596 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:05.932611 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:05.932652 (+ 41us) server_negotiation.cc:407] Connection header received
0618 22:33:05.934344 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:05.934366 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:05.934791 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:05.935147 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:05.936931 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:05.938613 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:05.940340 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:05.941023 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:05.941165 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:05.945663 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:05.945671 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:05.945687 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:05.945726 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:05.948936 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:05.949717 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:05.949723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:05.949729 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:05.949879 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:05.950318 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:05.950323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:05.950325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:05.950859 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:05.951088 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:05.951509 (+ 421us) server_negotiation.cc:299] Negotiation successful
0618 22:33:05.951855 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":331,"threads_started":1}
I20260618 22:33:05.956657 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:33:05.958629 22038 raft_consensus.cc:348] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00d5c4fff4e492ca92c800c4064a4ee" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40179 } }
I20260618 22:33:05.959069 22038 raft_consensus.cc:374] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:05.959159 22038 raft_consensus.cc:729] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00d5c4fff4e492ca92c800c4064a4ee, State: Initialized, Role: FOLLOWER
I20260618 22:33:05.959793 22038 consensus_queue.cc:260] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00d5c4fff4e492ca92c800c4064a4ee" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40179 } }
I20260618 22:33:05.960057 22038 raft_consensus.cc:388] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:05.960187 22038 raft_consensus.cc:482] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:05.960338 22038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:05.963245 22038 raft_consensus.cc:504] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00d5c4fff4e492ca92c800c4064a4ee" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40179 } }
I20260618 22:33:05.963824 22038 leader_election.cc:302] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e00d5c4fff4e492ca92c800c4064a4ee; no voters:
I20260618 22:33:05.964463 22038 leader_election.cc:288] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:05.964638 22044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:05.965656 22044 raft_consensus.cc:686] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [term 1 LEADER]: Becoming Leader. State: Replica: e00d5c4fff4e492ca92c800c4064a4ee, State: Running, Role: LEADER
I20260618 22:33:05.966259 22044 consensus_queue.cc:237] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00d5c4fff4e492ca92c800c4064a4ee" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40179 } }
I20260618 22:33:05.966612 22038 sys_catalog.cc:564] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:05.971549 22046 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader e00d5c4fff4e492ca92c800c4064a4ee. Latest consensus state: current_term: 1 leader_uuid: "e00d5c4fff4e492ca92c800c4064a4ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00d5c4fff4e492ca92c800c4064a4ee" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40179 } } }
I20260618 22:33:05.971949 22046 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:05.972543 22045 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e00d5c4fff4e492ca92c800c4064a4ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00d5c4fff4e492ca92c800c4064a4ee" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40179 } } }
I20260618 22:33:05.972822 22045 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:05.974678 22052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:05.979919 22052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:05.988950 22052 catalog_manager.cc:1347] Generated new cluster ID: 5adf2c0dd5314d0fa81ab292d9acc821
I20260618 22:33:05.989065 22052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:06.007311 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:06.000143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.18.150.190:40179)
0618 22:33:06.000343 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:06.000362 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:06.000896 (+ 534us) server_negotiation.cc:407] Connection header received
0618 22:33:06.001086 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:06.001091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:06.001156 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:06.001241 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:06.002281 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:06.003046 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:06.004293 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:06.004724 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:06.004839 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:06.005831 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:06.005835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:06.005838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:06.006532 (+ 694us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:06.006592 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:06.006623 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:06.006719 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:06.006838 (+ 119us) server_negotiation.cc:299] Negotiation successful
0618 22:33:06.006934 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":69}
I20260618 22:33:06.009840 22052 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:06.011606 22052 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:06.019940 22052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e00d5c4fff4e492ca92c800c4064a4ee: Generated new TSK 0
I20260618 22:33:06.020959 22052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:33:06.031705 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 21971
I20260618 22:33:06.032923 22033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:06.119594 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 21971
2026-06-18T22:33:06Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1782 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:33:06 dist-test-slave-h7jz krb5kdc[22072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:33:06 dist-test-slave-h7jz krb5kdc[22072](info): set up 2 sockets
Jun 18 22:33:06 dist-test-slave-h7jz krb5kdc[22072](info): commencing operation
krb5kdc: starting...
W20260618 22:33:07.101997 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:33:07 dist-test-slave-h7jz krb5kdc[22072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:33:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:33:07.278478 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33073
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:33073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:07.582813 22085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:07.583103 22085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:07.583191 22085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:07.590931 22085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:07.591058 22085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:07.591104 22085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:07.591158 22085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:07.591200 22085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:07.600931 22085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:33073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:07.602259 22085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:07.603606 22085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:07.615392 22090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:07.615434 22091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:07.615492 22093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:07.616122 22085 server_base.cc:1034] running on GCE node
I20260618 22:33:07.616883 22085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:07.618353 22085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:07.619581 22085 hybrid_clock.cc:648] HybridClock initialized: now 1781821987619505 us; error 61 us; skew 500 ppm
I20260618 22:33:07.619938 22085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:07 dist-test-slave-h7jz krb5kdc[22072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821987, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:07.623018 22085 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:33:07.624620 22085 webserver.cc:460] Webserver started at http://127.18.150.190:33317/ using document root <none> and password file <none>
I20260618 22:33:07.625288 22085 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:07.625408 22085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:07.625690 22085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:07.628624 22085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "e24c5e4e0fa2465d98c6e0998d1a2506"
format_stamp: "Formatted at 2026-06-18 22:33:07 on dist-test-slave-h7jz"
server_key: "e4442b26b2058dea89ef6abe8bf0308e"
server_key_iv: "321fe48fc43113f980b7bfadd91d0ae5"
server_key_version: "encryptionkey@0"
I20260618 22:33:07.629346 22085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "e24c5e4e0fa2465d98c6e0998d1a2506"
format_stamp: "Formatted at 2026-06-18 22:33:07 on dist-test-slave-h7jz"
server_key: "e4442b26b2058dea89ef6abe8bf0308e"
server_key_iv: "321fe48fc43113f980b7bfadd91d0ae5"
server_key_version: "encryptionkey@0"
I20260618 22:33:07.634539 22085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:33:07.637987 22100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:07.639365 22085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:33:07.639511 22085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "e24c5e4e0fa2465d98c6e0998d1a2506"
format_stamp: "Formatted at 2026-06-18 22:33:07 on dist-test-slave-h7jz"
server_key: "e4442b26b2058dea89ef6abe8bf0308e"
server_key_iv: "321fe48fc43113f980b7bfadd91d0ae5"
server_key_version: "encryptionkey@0"
I20260618 22:33:07.639693 22085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:07.652842 22085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:07.656661 22085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:07.656916 22085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:07.673542 22085 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:33073
I20260618 22:33:07.673576 22151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:33073 every 8 connection(s)
I20260618 22:33:07.674883 22085 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:07.678623 22152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:07.684473 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 22085
I20260618 22:33:07.684686 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:07.685429 19034 external_mini_cluster.cc:1428] Setting key ce6e010c982fa7c0a3c54094a1da1aa4
I20260618 22:33:07.689880 22152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506: Bootstrap starting.
I20260618 22:33:07.693380 22152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:07.694566 22152 log.cc:824] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:07.697741 22152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506: No bootstrap required, opened a new log
Jun 18 22:33:07 dist-test-slave-h7jz krb5kdc[22072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:07.707441 22152 raft_consensus.cc:348] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e24c5e4e0fa2465d98c6e0998d1a2506" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33073 } }
I20260618 22:33:07.707888 22152 raft_consensus.cc:374] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:07.708030 22152 raft_consensus.cc:729] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e24c5e4e0fa2465d98c6e0998d1a2506, State: Initialized, Role: FOLLOWER
I20260618 22:33:07.708711 22152 consensus_queue.cc:260] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e24c5e4e0fa2465d98c6e0998d1a2506" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33073 } }
I20260618 22:33:07.708967 22152 raft_consensus.cc:388] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:07.708998 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:07.687572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56212 (local address 127.18.150.190:33073)
0618 22:33:07.688340 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:07.688352 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:07.688392 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:33:07.690037 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:07.690066 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:07.690462 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:07.690785 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:07.692448 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:07.693820 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:07.695503 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:07.696289 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:07.696417 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:07.700795 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:07.700805 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:07.700829 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:07.700872 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:07.704638 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:07.705293 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:07.705302 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:07.705311 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:07.705520 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:07.705817 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:07.705822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:07.705825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:07.706545 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:07.706834 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:07.707150 (+ 316us) server_negotiation.cc:299] Negotiation successful
0618 22:33:07.707587 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":370,"threads_started":1}
I20260618 22:33:07.709128 22152 raft_consensus.cc:482] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:07.709311 22152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:07.711591 22152 raft_consensus.cc:504] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e24c5e4e0fa2465d98c6e0998d1a2506" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33073 } }
I20260618 22:33:07.712049 22152 leader_election.cc:302] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e24c5e4e0fa2465d98c6e0998d1a2506; no voters:
I20260618 22:33:07.712554 22152 leader_election.cc:288] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:07.712684 22157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:07.713724 22157 raft_consensus.cc:686] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [term 1 LEADER]: Becoming Leader. State: Replica: e24c5e4e0fa2465d98c6e0998d1a2506, State: Running, Role: LEADER
I20260618 22:33:07.714255 22157 consensus_queue.cc:237] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e24c5e4e0fa2465d98c6e0998d1a2506" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33073 } }
I20260618 22:33:07.714650 22152 sys_catalog.cc:564] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:07.717054 22159 sys_catalog.cc:455] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e24c5e4e0fa2465d98c6e0998d1a2506. Latest consensus state: current_term: 1 leader_uuid: "e24c5e4e0fa2465d98c6e0998d1a2506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e24c5e4e0fa2465d98c6e0998d1a2506" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33073 } } }
I20260618 22:33:07.717442 22159 sys_catalog.cc:458] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:07.718007 22158 sys_catalog.cc:455] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e24c5e4e0fa2465d98c6e0998d1a2506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e24c5e4e0fa2465d98c6e0998d1a2506" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33073 } } }
I20260618 22:33:07.718294 22158 sys_catalog.cc:458] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:07.723059 22166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:07.732357 22166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:07.741851 22166 catalog_manager.cc:1347] Generated new cluster ID: 068eb753e73747b994fa76584a14b832
I20260618 22:33:07.741993 22166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:07.795861 22166 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:07.797775 22166 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:07.808770 22166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e24c5e4e0fa2465d98c6e0998d1a2506: Generated new TSK 0
I20260618 22:33:07.809923 22166 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:33:07.857298 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:33073
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:08.169621 22178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:08.169934 22178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:08.170084 22178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:08.177219 22178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:08.177346 22178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:08.177502 22178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:33:08.187220 22178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:33073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:08.188570 22178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:08.189911 22178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:08.202615 22184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:08.202687 22186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:08.202652 22183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:08.203456 22178 server_base.cc:1034] running on GCE node
I20260618 22:33:08.204033 22178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:08.204981 22178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:08.206171 22178 hybrid_clock.cc:648] HybridClock initialized: now 1781821988206124 us; error 42 us; skew 500 ppm
I20260618 22:33:08.206463 22178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:08 dist-test-slave-h7jz krb5kdc[22072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821988, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:08.209451 22178 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:33:08.210812 22178 webserver.cc:460] Webserver started at http://127.18.150.129:38621/ using document root <none> and password file <none>
I20260618 22:33:08.211442 22178 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:08.211553 22178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:08.211829 22178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:08.214653 22178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "17d7d589dfb54398b48b04dab6c7eb8f"
format_stamp: "Formatted at 2026-06-18 22:33:08 on dist-test-slave-h7jz"
server_key: "a7440d90912d6aa19cc76bd7d917409d"
server_key_iv: "9bcb6694360c5df67c93978a1d7ec2c1"
server_key_version: "encryptionkey@0"
I20260618 22:33:08.215401 22178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "17d7d589dfb54398b48b04dab6c7eb8f"
format_stamp: "Formatted at 2026-06-18 22:33:08 on dist-test-slave-h7jz"
server_key: "a7440d90912d6aa19cc76bd7d917409d"
server_key_iv: "9bcb6694360c5df67c93978a1d7ec2c1"
server_key_version: "encryptionkey@0"
I20260618 22:33:08.220382 22178 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:33:08.223768 22193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:08.225147 22178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:33:08.225297 22178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "17d7d589dfb54398b48b04dab6c7eb8f"
format_stamp: "Formatted at 2026-06-18 22:33:08 on dist-test-slave-h7jz"
server_key: "a7440d90912d6aa19cc76bd7d917409d"
server_key_iv: "9bcb6694360c5df67c93978a1d7ec2c1"
server_key_version: "encryptionkey@0"
I20260618 22:33:08.225512 22178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:08.253212 22178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:08.257459 22178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:08.257742 22178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:08.258770 22178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:08.260082 22178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:08.260159 22178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:08.260275 22178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:08.260327 22178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:08.289736 22178 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:40233
I20260618 22:33:08.289772 22305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:40233 every 8 connection(s)
I20260618 22:33:08.290988 22178 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:33:08.295740 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 22178
I20260618 22:33:08.295920 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:33:08.296494 19034 external_mini_cluster.cc:1428] Setting key 8d6e27babb07408bb6ed41fdf33d6ab7
Jun 18 22:33:08 dist-test-slave-h7jz krb5kdc[22072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821988, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:08.311427 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:08.293460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:49315 (local address 127.18.150.190:33073)
0618 22:33:08.293859 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:08.293864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:08.294978 (+ 1114us) server_negotiation.cc:407] Connection header received
0618 22:33:08.296820 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:08.296827 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:08.296911 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:08.297035 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:08.298779 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:08.299482 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:08.300992 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:08.301383 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:08.301485 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:08.305018 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:08.305025 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:08.305030 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:08.305070 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:08.308099 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:08.308973 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:08.308979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:08.308982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:08.309133 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:08.309527 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:08.309532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:08.309535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:08.309896 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:08.310095 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:08.310800 (+ 705us) server_negotiation.cc:299] Negotiation successful
0618 22:33:08.311042 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":203,"threads_started":1}
I20260618 22:33:08.312245 22309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:08.294073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33073 (local address 127.18.150.129:49315)
0618 22:33:08.294809 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:08.294850 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:08.296531 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:08.297391 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:08.297403 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:08.298001 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:08.298618 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:08.298637 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:08.299623 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:08.299627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:08.300862 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:08.300874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:08.301586 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:08.301593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:08.301818 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:08.302608 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:08.302632 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:08.304773 (+ 2141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:08.308268 (+ 3495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:08.308297 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:08.308316 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:08.308810 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:08.309236 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:08.309239 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:08.309242 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:08.309425 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:08.310070 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:08.310077 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:08.310432 (+ 355us) client_negotiation.cc:769] Sending connection context
0618 22:33:08.310659 (+ 227us) client_negotiation.cc:240] Negotiation successful
0618 22:33:08.311053 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":277,"threads_started":1}
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
I20260618 22:33:08.313889 22306 heartbeater.cc:347] Connected to a master server at 127.18.150.190:33073
I20260618 22:33:08.314203 22306 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:08.314980 22306 heartbeater.cc:511] Master 127.18.150.190:33073 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
I20260618 22:33:08.317190 22117 ts_manager.cc:194] Registered new tserver with Master: 17d7d589dfb54398b48b04dab6c7eb8f (127.18.150.129:40233)
I20260618 22:33:08.318729 22117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:49315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:33:08.338078 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:33073
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:08.637678 22312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:08.637988 22312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:08.638134 22312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:08.645419 22312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:08.645519 22312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:08.645655 22312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:33:08.655316 22312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:33073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:08.656548 22312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:08.657789 22312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:08.669929 22317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:08.669989 22318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:08.669899 22320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:08.670993 22312 server_base.cc:1034] running on GCE node
I20260618 22:33:08.671584 22312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:08.672477 22312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:08.673662 22312 hybrid_clock.cc:648] HybridClock initialized: now 1781821988673612 us; error 45 us; skew 500 ppm
I20260618 22:33:08.673974 22312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:08 dist-test-slave-h7jz krb5kdc[22072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821988, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:08.681113 22312 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:33:08.682726 22312 webserver.cc:460] Webserver started at http://127.18.150.130:37291/ using document root <none> and password file <none>
I20260618 22:33:08.683378 22312 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:08.683495 22312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:08.683775 22312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:08.686563 22312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "f29fe82b256143768988ee7c1e54f5fa"
format_stamp: "Formatted at 2026-06-18 22:33:08 on dist-test-slave-h7jz"
server_key: "2b006cd24527d13c7ac788b94fdd6727"
server_key_iv: "79fd01c3e8a77645ffe07345f94aad20"
server_key_version: "encryptionkey@0"
I20260618 22:33:08.687276 22312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "f29fe82b256143768988ee7c1e54f5fa"
format_stamp: "Formatted at 2026-06-18 22:33:08 on dist-test-slave-h7jz"
server_key: "2b006cd24527d13c7ac788b94fdd6727"
server_key_iv: "79fd01c3e8a77645ffe07345f94aad20"
server_key_version: "encryptionkey@0"
I20260618 22:33:08.692094 22312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:33:08.695346 22327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:08.696563 22312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:33:08.696710 22312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "f29fe82b256143768988ee7c1e54f5fa"
format_stamp: "Formatted at 2026-06-18 22:33:08 on dist-test-slave-h7jz"
server_key: "2b006cd24527d13c7ac788b94fdd6727"
server_key_iv: "79fd01c3e8a77645ffe07345f94aad20"
server_key_version: "encryptionkey@0"
I20260618 22:33:08.696882 22312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:08.720422 22312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:08.724267 22312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:08.724527 22312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:08.725451 22312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:08.726801 22312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:08.726876 22312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:08.726986 22312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:08.727035 22312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:08.756309 22312 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:37591
I20260618 22:33:08.756347 22439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:37591 every 8 connection(s)
I20260618 22:33:08.757496 22312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:33:08.763823 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 22312
I20260618 22:33:08.763991 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:33:08.764525 19034 external_mini_cluster.cc:1428] Setting key 012a46f86f0dfb1650eda29365f74d0d
Jun 18 22:33:08 dist-test-slave-h7jz krb5kdc[22072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821988, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:08.776599 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:08.759826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:56211 (local address 127.18.150.190:33073)
0618 22:33:08.760009 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:08.760015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:08.761201 (+ 1186us) server_negotiation.cc:407] Connection header received
0618 22:33:08.762920 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:08.762924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:08.762997 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:08.763083 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:08.764918 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:08.765715 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:08.767453 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:08.768055 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:08.768198 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:08.771419 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:08.771424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:08.771428 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:08.771457 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:08.773795 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:08.774608 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:08.774612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:08.774614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:08.774727 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:08.775121 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:08.775125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:08.775127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:08.775349 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:08.775447 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:08.776210 (+ 763us) server_negotiation.cc:299] Negotiation successful
0618 22:33:08.776365 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:33:08.778016 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:08.760385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33073 (local address 127.18.150.130:56211)
0618 22:33:08.761049 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:08.761086 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:08.762688 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:08.763262 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:08.763269 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:08.763942 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:08.764755 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:08.764782 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:08.765855 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:08.765860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:08.767304 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:08.767316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:08.768250 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:08.768256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:08.768439 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:08.769072 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:08.769093 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:08.771221 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:08.773933 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:08.773968 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:08.773986 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:08.774507 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:08.774812 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:08.774816 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:08.774820 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:08.775028 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:08.775461 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:08.775468 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:08.775856 (+ 388us) client_negotiation.cc:769] Sending connection context
0618 22:33:08.776126 (+ 270us) client_negotiation.cc:240] Negotiation successful
0618 22:33:08.776620 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":254,"threads_started":1}
I20260618 22:33:08.779400 22440 heartbeater.cc:347] Connected to a master server at 127.18.150.190:33073
I20260618 22:33:08.779706 22440 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:08.780382 22440 heartbeater.cc:511] Master 127.18.150.190:33073 requested a full tablet report, sending...
I20260618 22:33:08.781738 22117 ts_manager.cc:194] Registered new tserver with Master: f29fe82b256143768988ee7c1e54f5fa (127.18.150.130:37591)
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:33:08.782550 22117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:56211
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:33:08.805454 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:33073
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:09.110160 22445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:09.110455 22445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:09.110584 22445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:09.117769 22445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:09.117897 22445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:09.118096 22445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:33:09.127689 22445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:33073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:09.128997 22445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:09.130380 22445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:09.142516 22453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:09.142560 22451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:09.142532 22450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:09.143402 22445 server_base.cc:1034] running on GCE node
I20260618 22:33:09.143944 22445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:09.144843 22445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:09.146035 22445 hybrid_clock.cc:648] HybridClock initialized: now 1781821989145984 us; error 49 us; skew 500 ppm
I20260618 22:33:09.146339 22445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:09 dist-test-slave-h7jz krb5kdc[22072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821989, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:09.149190 22445 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:33:09.150504 22445 webserver.cc:460] Webserver started at http://127.18.150.131:33633/ using document root <none> and password file <none>
I20260618 22:33:09.151106 22445 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:09.151199 22445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:09.151434 22445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:09.154150 22445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "79d93db631364208aa07165228036188"
format_stamp: "Formatted at 2026-06-18 22:33:09 on dist-test-slave-h7jz"
server_key: "2b5ea0e911a2946bcb2d8e31b86b9f3c"
server_key_iv: "d2c03cc49786dc742f18e8409adfe8b9"
server_key_version: "encryptionkey@0"
I20260618 22:33:09.154836 22445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "79d93db631364208aa07165228036188"
format_stamp: "Formatted at 2026-06-18 22:33:09 on dist-test-slave-h7jz"
server_key: "2b5ea0e911a2946bcb2d8e31b86b9f3c"
server_key_iv: "d2c03cc49786dc742f18e8409adfe8b9"
server_key_version: "encryptionkey@0"
I20260618 22:33:09.159327 22445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:33:09.162452 22460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:09.163694 22445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:33:09.163808 22445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "79d93db631364208aa07165228036188"
format_stamp: "Formatted at 2026-06-18 22:33:09 on dist-test-slave-h7jz"
server_key: "2b5ea0e911a2946bcb2d8e31b86b9f3c"
server_key_iv: "d2c03cc49786dc742f18e8409adfe8b9"
server_key_version: "encryptionkey@0"
I20260618 22:33:09.163949 22445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:09.176415 22445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:09.179878 22445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:09.180106 22445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:09.180884 22445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:09.182129 22445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:09.182199 22445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:09.182281 22445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:09.182323 22445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:09.210930 22445 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:40689
I20260618 22:33:09.210966 22572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:40689 every 8 connection(s)
I20260618 22:33:09.212188 22445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:33:09.221257 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 22445
I20260618 22:33:09.221398 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:33:09.221823 19034 external_mini_cluster.cc:1428] Setting key 01748ac33b88be41e107a41b9241b516
Jun 18 22:33:09 dist-test-slave-h7jz krb5kdc[22072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821989, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:09.231499 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:09.214783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:60677 (local address 127.18.150.190:33073)
0618 22:33:09.214986 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:09.214992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:09.216185 (+ 1193us) server_negotiation.cc:407] Connection header received
0618 22:33:09.218010 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:09.218017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:09.218097 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:09.218178 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:09.219666 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.220383 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.221987 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.222513 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.222646 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:09.226359 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:09.226364 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:09.226368 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:09.226402 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:09.228790 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:09.229555 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:09.229559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:09.229561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:09.229685 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:09.230066 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:09.230070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:09.230074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:09.230350 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:09.230488 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:09.231098 (+ 610us) server_negotiation.cc:299] Negotiation successful
0618 22:33:09.231259 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260618 22:33:09.232474 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:09.215324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33073 (local address 127.18.150.131:60677)
0618 22:33:09.216042 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:09.216082 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:09.217691 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:09.218395 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:09.218404 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:09.218930 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:09.219507 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:09.219525 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.220508 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.220511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:09.221804 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:09.221820 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.222718 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.222723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:09.222992 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:09.223689 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:09.223714 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:09.226116 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:09.228956 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:09.228963 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:09.228976 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:09.229434 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:09.229771 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:09.229775 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:09.229777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:09.229984 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:09.230466 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:09.230472 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:09.230793 (+ 321us) client_negotiation.cc:769] Sending connection context
0618 22:33:09.231022 (+ 229us) client_negotiation.cc:240] Negotiation successful
0618 22:33:09.231418 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":262,"threads_started":1}
I20260618 22:33:09.233831 22573 heartbeater.cc:347] Connected to a master server at 127.18.150.190:33073
I20260618 22:33:09.234150 22573 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:09.234843 22573 heartbeater.cc:511] Master 127.18.150.190:33073 requested a full tablet report, sending...
I20260618 22:33:09.236306 22116 ts_manager.cc:194] Registered new tserver with Master: 79d93db631364208aa07165228036188 (127.18.150.131:40689)
I20260618 22:33:09.237200 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:33:09.237407 22116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:60677
Jun 18 22:33:09 dist-test-slave-h7jz krb5kdc[22072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821989, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 18 22:33:09 dist-test-slave-h7jz krb5kdc[22072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821989, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:09.282193 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:09.268917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40002 (local address 127.18.150.190:33073)
0618 22:33:09.269109 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:09.269116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:09.269499 (+ 383us) server_negotiation.cc:407] Connection header received
0618 22:33:09.269682 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:09.269687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:09.269751 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:09.269833 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:09.271175 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.271893 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.273118 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.273525 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.273633 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:09.277311 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:09.277316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:09.277319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:09.277353 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:09.279756 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:09.280333 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:09.280337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:09.280340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:09.280453 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:09.280823 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:09.280828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:09.280831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:09.281146 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:09.281363 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:09.281665 (+ 302us) server_negotiation.cc:299] Negotiation successful
0618 22:33:09.281871 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260618 22:33:09.282728 22116 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40002
I20260618 22:33:09.284116 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 22178
Jun 18 22:33:09 dist-test-slave-h7jz krb5kdc[22072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821989, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:33:09.299204 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:09.285048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35598 (local address 127.18.150.129:40233)
0618 22:33:09.285588 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:09.285595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:09.285617 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:33:09.285670 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:09.285676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:09.285877 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:09.286035 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:09.287123 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.288085 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.289276 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.289847 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.289986 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:09.294119 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:09.294129 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:09.294142 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:09.294178 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:09.296798 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:09.297375 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:09.297382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:09.297384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:09.297487 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:09.297804 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:09.297807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:09.297809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:09.298164 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:09.298375 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:09.298703 (+ 328us) server_negotiation.cc:299] Negotiation successful
0618 22:33:09.298900 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":262,"threads_started":1}
W20260618 22:33:09.300444 22301 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35598
W20260618 22:33:09.301630 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 22178: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:33:09.301816 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 22178
I20260618 22:33:09.313935 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 22312
Jun 18 22:33:09 dist-test-slave-h7jz krb5kdc[22072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821989, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:33:09.331297 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:09.314873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44216 (local address 127.18.150.130:37591)
0618 22:33:09.315495 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:09.315503 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:09.315530 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:33:09.315590 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:09.315597 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:09.315841 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:09.315971 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:09.317252 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.318366 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.319633 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.320245 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.320356 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:09.324706 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:09.324724 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:09.324745 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:09.324801 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:09.328512 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:09.329262 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:09.329271 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:09.329275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:09.329427 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:09.329725 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:09.329728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:09.329730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:09.330126 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:09.330348 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:09.330703 (+ 355us) server_negotiation.cc:299] Negotiation successful
0618 22:33:09.330947 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":276,"threads_started":1}
W20260618 22:33:09.332576 22435 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44216
W20260618 22:33:09.334129 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 22312: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:33:09.334278 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 22312
I20260618 22:33:09.347179 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 22445
Jun 18 22:33:09 dist-test-slave-h7jz krb5kdc[22072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821989, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:33:09.362998 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:09.348224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50268 (local address 127.18.150.131:40689)
0618 22:33:09.348915 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:09.348922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:09.348951 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:33:09.349009 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:09.349015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:09.349232 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:09.349376 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:09.350797 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.351892 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.353128 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:09.353696 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:09.353814 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:09.357862 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:09.357875 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:09.357890 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:09.357924 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:09.360521 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:09.361161 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:09.361167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:09.361170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:09.361279 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:09.361589 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:09.361592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:09.361594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:09.361980 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:09.362221 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:09.362491 (+ 270us) server_negotiation.cc:299] Negotiation successful
0618 22:33:09.362716 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":338,"threads_started":1}
W20260618 22:33:09.364337 22568 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50268
W20260618 22:33:09.365449 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 22445: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:33:09.365610 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 22445
I20260618 22:33:09.377377 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 22085
I20260618 22:33:09.379083 22147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:09.480625 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 22085
2026-06-18T22:33:09Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3365 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:33:09 dist-test-slave-h7jz krb5kdc[22590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:33:09 dist-test-slave-h7jz krb5kdc[22590](info): set up 2 sockets
Jun 18 22:33:09 dist-test-slave-h7jz krb5kdc[22590](info): commencing operation
krb5kdc: starting...
W20260618 22:33:10.469992 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:33:10 dist-test-slave-h7jz krb5kdc[22590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:33:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:33:10.645285 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39547
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:39547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:10.950474 22603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:10.950798 22603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:10.950922 22603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:10.957954 22603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:10.958065 22603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:10.958144 22603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:10.958223 22603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:10.958287 22603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:10.967700 22603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:39547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:10.968914 22603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:10.970216 22603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:10.980957 22611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:10.980921 22609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:10.981009 22608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:10.981884 22603 server_base.cc:1034] running on GCE node
I20260618 22:33:10.982669 22603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:10.983997 22603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:10.985203 22603 hybrid_clock.cc:648] HybridClock initialized: now 1781821990985152 us; error 37 us; skew 500 ppm
I20260618 22:33:10.985520 22603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:10 dist-test-slave-h7jz krb5kdc[22590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821990, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:10.988407 22603 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:33:10.989717 22603 webserver.cc:460] Webserver started at http://127.18.150.190:32915/ using document root <none> and password file <none>
I20260618 22:33:10.990391 22603 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:10.990499 22603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:10.990770 22603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:10.993541 22603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "58df6973e1b343488cb7f825ce758fcd"
format_stamp: "Formatted at 2026-06-18 22:33:10 on dist-test-slave-h7jz"
server_key: "1fcab396176a18fb7fb7304fbcdb29ad"
server_key_iv: "097a017857dbde1c4e08a1ad2f1508d5"
server_key_version: "encryptionkey@0"
I20260618 22:33:10.994302 22603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "58df6973e1b343488cb7f825ce758fcd"
format_stamp: "Formatted at 2026-06-18 22:33:10 on dist-test-slave-h7jz"
server_key: "1fcab396176a18fb7fb7304fbcdb29ad"
server_key_iv: "097a017857dbde1c4e08a1ad2f1508d5"
server_key_version: "encryptionkey@0"
I20260618 22:33:10.998994 22603 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260618 22:33:11.002161 22618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:11.003441 22603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:33:11.003602 22603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "58df6973e1b343488cb7f825ce758fcd"
format_stamp: "Formatted at 2026-06-18 22:33:10 on dist-test-slave-h7jz"
server_key: "1fcab396176a18fb7fb7304fbcdb29ad"
server_key_iv: "097a017857dbde1c4e08a1ad2f1508d5"
server_key_version: "encryptionkey@0"
I20260618 22:33:11.003770 22603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:11.016929 22603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:11.020874 22603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:11.021149 22603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:11.036839 22603 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:39547
I20260618 22:33:11.036855 22669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:39547 every 8 connection(s)
I20260618 22:33:11.038177 22603 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:11.040280 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 22603
I20260618 22:33:11.040473 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:11.041045 19034 external_mini_cluster.cc:1428] Setting key 35e099bc3d4032d1559d1a6596f10387
I20260618 22:33:11.041613 22670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:11.054878 22670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd: Bootstrap starting.
Jun 18 22:33:11 dist-test-slave-h7jz krb5kdc[22590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:11.057857 22670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:11.059149 22670 log.cc:824] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:11.062307 22670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd: No bootstrap required, opened a new log
I20260618 22:33:11.064181 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:11.042831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50902 (local address 127.18.150.190:39547)
0618 22:33:11.043531 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:11.043541 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:11.043580 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:33:11.045849 (+ 2269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:11.045870 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:11.046318 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:11.046658 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:11.048401 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:11.049630 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:11.051361 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:11.052032 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:11.052163 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:11.056443 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:11.056450 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:11.056464 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:11.056502 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:11.059843 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:11.060631 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:11.060639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:11.060647 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:11.060843 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:11.061194 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:11.061199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:11.061203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:11.061847 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:11.062128 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:11.062511 (+ 383us) server_negotiation.cc:299] Negotiation successful
0618 22:33:11.062939 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":310,"threads_started":1}
I20260618 22:33:11.070513 22670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58df6973e1b343488cb7f825ce758fcd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39547 } }
I20260618 22:33:11.070815 22670 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:11.070883 22670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58df6973e1b343488cb7f825ce758fcd, State: Initialized, Role: FOLLOWER
I20260618 22:33:11.071365 22670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58df6973e1b343488cb7f825ce758fcd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39547 } }
I20260618 22:33:11.071616 22670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:11.071746 22670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:11.071903 22670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:11.074127 22670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58df6973e1b343488cb7f825ce758fcd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39547 } }
I20260618 22:33:11.074546 22670 leader_election.cc:302] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58df6973e1b343488cb7f825ce758fcd; no voters:
I20260618 22:33:11.075011 22670 leader_election.cc:288] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:11.075135 22675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:11.075726 22675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [term 1 LEADER]: Becoming Leader. State: Replica: 58df6973e1b343488cb7f825ce758fcd, State: Running, Role: LEADER
I20260618 22:33:11.076206 22675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58df6973e1b343488cb7f825ce758fcd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39547 } }
I20260618 22:33:11.076802 22670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:11.078861 22676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58df6973e1b343488cb7f825ce758fcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58df6973e1b343488cb7f825ce758fcd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39547 } } }
I20260618 22:33:11.079222 22676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:11.078850 22677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58df6973e1b343488cb7f825ce758fcd. Latest consensus state: current_term: 1 leader_uuid: "58df6973e1b343488cb7f825ce758fcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58df6973e1b343488cb7f825ce758fcd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39547 } } }
I20260618 22:33:11.079684 22677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:11.080957 22682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:11.085350 22682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:11.093544 22682 catalog_manager.cc:1347] Generated new cluster ID: 228e6e6d451c47bfb0e3447c22b65569
I20260618 22:33:11.093663 22682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:11.114184 22682 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:11.115957 22682 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:11.132963 22682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58df6973e1b343488cb7f825ce758fcd: Generated new TSK 0
I20260618 22:33:11.134045 22682 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:33:11.175788 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39547
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:11.476016 22696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:11.476269 22696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:11.476392 22696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:11.483568 22696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:11.483675 22696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:11.483824 22696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:33:11.493199 22696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:11.494446 22696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:11.495728 22696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:11.507751 22704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:11.507786 22702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:11.508195 22701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:11.508603 22696 server_base.cc:1034] running on GCE node
I20260618 22:33:11.509140 22696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:11.510020 22696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:11.511207 22696 hybrid_clock.cc:648] HybridClock initialized: now 1781821991511162 us; error 42 us; skew 500 ppm
I20260618 22:33:11.511482 22696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:11 dist-test-slave-h7jz krb5kdc[22590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821991, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:11.514297 22696 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:33:11.515636 22696 webserver.cc:460] Webserver started at http://127.18.150.129:34131/ using document root <none> and password file <none>
I20260618 22:33:11.516274 22696 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:11.516382 22696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:11.516681 22696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:11.519439 22696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "207198f06a5c4f1582a1595563a01e5e"
format_stamp: "Formatted at 2026-06-18 22:33:11 on dist-test-slave-h7jz"
server_key: "2d095f8dd3b76ebef53db8d4b4abb827"
server_key_iv: "3485d52bb890114fe17dfae110183ca1"
server_key_version: "encryptionkey@0"
I20260618 22:33:11.520159 22696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "207198f06a5c4f1582a1595563a01e5e"
format_stamp: "Formatted at 2026-06-18 22:33:11 on dist-test-slave-h7jz"
server_key: "2d095f8dd3b76ebef53db8d4b4abb827"
server_key_iv: "3485d52bb890114fe17dfae110183ca1"
server_key_version: "encryptionkey@0"
I20260618 22:33:11.524966 22696 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260618 22:33:11.528168 22711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:11.529407 22696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260618 22:33:11.529538 22696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "207198f06a5c4f1582a1595563a01e5e"
format_stamp: "Formatted at 2026-06-18 22:33:11 on dist-test-slave-h7jz"
server_key: "2d095f8dd3b76ebef53db8d4b4abb827"
server_key_iv: "3485d52bb890114fe17dfae110183ca1"
server_key_version: "encryptionkey@0"
I20260618 22:33:11.529724 22696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:11.559630 22696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:11.563649 22696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:11.563910 22696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:11.564826 22696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:11.566179 22696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:11.566252 22696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:11.566361 22696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:11.566417 22696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:11.595407 22696 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:45459
I20260618 22:33:11.595472 22823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:45459 every 8 connection(s)
I20260618 22:33:11.596626 22696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:33:11.602869 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 22696
I20260618 22:33:11.603055 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:33:11.603607 19034 external_mini_cluster.cc:1428] Setting key 072375a7f99d4494df1792fe9e81920d
Jun 18 22:33:11 dist-test-slave-h7jz krb5kdc[22590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821991, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:11.616168 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:11.599083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:43665 (local address 127.18.150.190:39547)
0618 22:33:11.599486 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:11.599490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:11.600505 (+ 1015us) server_negotiation.cc:407] Connection header received
0618 22:33:11.602322 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:11.602328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:11.602407 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:11.602520 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:11.604034 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:11.604962 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:11.606555 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:11.606977 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:11.607093 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:11.610884 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:11.610891 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:11.610895 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:11.610933 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:11.613286 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:11.614093 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:11.614096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:11.614099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:11.614205 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:11.614619 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:11.614622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:11.614624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:11.614879 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:11.614977 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:11.615758 (+ 781us) server_negotiation.cc:299] Negotiation successful
0618 22:33:11.615912 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":198,"threads_started":1}
I20260618 22:33:11.617667 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:11.599662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39547 (local address 127.18.150.129:43665)
0618 22:33:11.600339 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:11.600386 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:11.602040 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:11.602747 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:11.602756 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:11.603307 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:11.603881 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:11.603899 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:11.605112 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:11.605118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:11.606428 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:11.606442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:11.607211 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:11.607219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:11.607485 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:11.608167 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:11.608193 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:11.610656 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:11.613417 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:11.613427 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:11.613445 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:11.613984 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:11.614298 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:11.614303 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:11.614306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:11.614532 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:11.614979 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:11.614986 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:11.615399 (+ 413us) client_negotiation.cc:769] Sending connection context
0618 22:33:11.615689 (+ 290us) client_negotiation.cc:240] Negotiation successful
0618 22:33:11.616207 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":267,"threads_started":1}
I20260618 22:33:11.619175 22824 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39547
I20260618 22:33:11.619498 22824 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:11.620360 22824 heartbeater.cc:511] Master 127.18.150.190:39547 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
I20260618 22:33:11.622570 22635 ts_manager.cc:194] Registered new tserver with Master: 207198f06a5c4f1582a1595563a01e5e (127.18.150.129:45459)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
I20260618 22:33:11.624068 22635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:43665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:33:11.644409 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39547
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:11.950248 22830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:11.950596 22830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:11.950767 22830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:11.960340 22830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:11.960443 22830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:11.960561 22830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:33:11.970175 22830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:11.971366 22830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:11.972584 22830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:11.985266 22835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:11.985240 22838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:11.985230 22836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:11.985987 22830 server_base.cc:1034] running on GCE node
I20260618 22:33:11.986590 22830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:11.987449 22830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:11.988657 22830 hybrid_clock.cc:648] HybridClock initialized: now 1781821991988598 us; error 51 us; skew 500 ppm
I20260618 22:33:11.988948 22830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:11 dist-test-slave-h7jz krb5kdc[22590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821991, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:11.991829 22830 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:33:11.993074 22830 webserver.cc:460] Webserver started at http://127.18.150.130:44729/ using document root <none> and password file <none>
I20260618 22:33:11.993712 22830 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:11.993803 22830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:11.994077 22830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:11.996726 22830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "47f62381cbcc48c4b4a911157a9b3cdc"
format_stamp: "Formatted at 2026-06-18 22:33:11 on dist-test-slave-h7jz"
server_key: "d34eb3055ab0af85450fccc503ad5be9"
server_key_iv: "7409b65a13865a8d175d12c89f7cc7a1"
server_key_version: "encryptionkey@0"
I20260618 22:33:11.997431 22830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "47f62381cbcc48c4b4a911157a9b3cdc"
format_stamp: "Formatted at 2026-06-18 22:33:11 on dist-test-slave-h7jz"
server_key: "d34eb3055ab0af85450fccc503ad5be9"
server_key_iv: "7409b65a13865a8d175d12c89f7cc7a1"
server_key_version: "encryptionkey@0"
I20260618 22:33:12.002202 22830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260618 22:33:12.005396 22845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:12.006675 22830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:12.006834 22830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "47f62381cbcc48c4b4a911157a9b3cdc"
format_stamp: "Formatted at 2026-06-18 22:33:11 on dist-test-slave-h7jz"
server_key: "d34eb3055ab0af85450fccc503ad5be9"
server_key_iv: "7409b65a13865a8d175d12c89f7cc7a1"
server_key_version: "encryptionkey@0"
I20260618 22:33:12.007001 22830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:12.027760 22830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:12.031595 22830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:12.031841 22830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:12.032903 22830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:12.034298 22830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:12.034387 22830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:12.034497 22830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:12.034540 22830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:12.064627 22830 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:45527
I20260618 22:33:12.064656 22957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:45527 every 8 connection(s)
I20260618 22:33:12.065827 22830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:33:12.070556 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 22830
I20260618 22:33:12.070726 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:33:12.071125 19034 external_mini_cluster.cc:1428] Setting key f964992f709a85af6f25e6ef298771c3
Jun 18 22:33:12 dist-test-slave-h7jz krb5kdc[22590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821991, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:12.085872 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:12.068227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:43371 (local address 127.18.150.190:39547)
0618 22:33:12.068390 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:12.068394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:12.069509 (+ 1115us) server_negotiation.cc:407] Connection header received
0618 22:33:12.071400 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:12.071406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:12.071487 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:12.071576 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:12.073068 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.073750 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.075816 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.076208 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.076297 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:12.079887 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:12.079893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:12.079898 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:12.079934 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:12.083001 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.083687 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.083693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.083696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.083832 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.084219 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.084224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.084228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.084536 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:12.084678 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:12.085309 (+ 631us) server_negotiation.cc:299] Negotiation successful
0618 22:33:12.085526 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:33:12.086719 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:12.068707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39547 (local address 127.18.150.130:43371)
0618 22:33:12.069361 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:12.069403 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:12.071134 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:12.071792 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:12.071800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:12.072291 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:12.072895 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:12.072928 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.073896 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.073902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:12.075689 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:12.075704 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.076424 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.076430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:12.076695 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:12.077334 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:12.077360 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:12.079684 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:12.083130 (+ 3446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:12.083137 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:12.083150 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:12.083565 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:12.083926 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:12.083929 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:12.083931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:12.084096 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:12.084639 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:12.084645 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:12.084946 (+ 301us) client_negotiation.cc:769] Sending connection context
0618 22:33:12.085191 (+ 245us) client_negotiation.cc:240] Negotiation successful
0618 22:33:12.085600 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":250,"threads_started":1}
I20260618 22:33:12.088081 22958 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39547
I20260618 22:33:12.088407 22958 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:12.089268 22958 heartbeater.cc:511] Master 127.18.150.190:39547 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:33:12.090679 22635 ts_manager.cc:194] Registered new tserver with Master: 47f62381cbcc48c4b4a911157a9b3cdc (127.18.150.130:45527)
I20260618 22:33:12.091490 22635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:43371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:33:12.110854 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39547
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:12.408411 22963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:12.408672 22963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:12.408794 22963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:12.415832 22963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:12.415937 22963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:12.416091 22963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:33:12.425540 22963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:12.426795 22963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:12.428045 22963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:12.440474 22968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:12.440428 22969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:12.440451 22971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:12.441215 22963 server_base.cc:1034] running on GCE node
I20260618 22:33:12.441805 22963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:12.442705 22963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:12.443912 22963 hybrid_clock.cc:648] HybridClock initialized: now 1781821992443859 us; error 43 us; skew 500 ppm
I20260618 22:33:12.444223 22963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:12 dist-test-slave-h7jz krb5kdc[22590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821992, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:12.447207 22963 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:33:12.448522 22963 webserver.cc:460] Webserver started at http://127.18.150.131:38519/ using document root <none> and password file <none>
I20260618 22:33:12.449179 22963 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:12.449280 22963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:12.449548 22963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:12.452318 22963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "b5066a5b92154ef9b6df2ea287a5bd88"
format_stamp: "Formatted at 2026-06-18 22:33:12 on dist-test-slave-h7jz"
server_key: "f6a52264b95f7acc3d238782dfedd8d4"
server_key_iv: "686c6cf71cf1c74febdef3eb83408bea"
server_key_version: "encryptionkey@0"
I20260618 22:33:12.453068 22963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "b5066a5b92154ef9b6df2ea287a5bd88"
format_stamp: "Formatted at 2026-06-18 22:33:12 on dist-test-slave-h7jz"
server_key: "f6a52264b95f7acc3d238782dfedd8d4"
server_key_iv: "686c6cf71cf1c74febdef3eb83408bea"
server_key_version: "encryptionkey@0"
I20260618 22:33:12.457911 22963 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260618 22:33:12.461077 22978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:12.462332 22963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:12.462464 22963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "b5066a5b92154ef9b6df2ea287a5bd88"
format_stamp: "Formatted at 2026-06-18 22:33:12 on dist-test-slave-h7jz"
server_key: "f6a52264b95f7acc3d238782dfedd8d4"
server_key_iv: "686c6cf71cf1c74febdef3eb83408bea"
server_key_version: "encryptionkey@0"
I20260618 22:33:12.462637 22963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:12.484663 22963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:12.488828 22963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:12.489101 22963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:12.490099 22963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:12.491393 22963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:12.491467 22963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:12.491587 22963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:12.491654 22963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:12.520504 22963 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:44987
I20260618 22:33:12.520586 23090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:44987 every 8 connection(s)
I20260618 22:33:12.521719 22963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:33:12.525871 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 22963
I20260618 22:33:12.526095 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:33:12.526527 19034 external_mini_cluster.cc:1428] Setting key dc8f084e937550e61709ada8f5c7f2fe
Jun 18 22:33:12 dist-test-slave-h7jz krb5kdc[22590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821992, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:12.541036 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:12.524233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:46685 (local address 127.18.150.190:39547)
0618 22:33:12.524427 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:12.524434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:12.525745 (+ 1311us) server_negotiation.cc:407] Connection header received
0618 22:33:12.527710 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:12.527715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:12.527803 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:12.527904 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:12.529783 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.530581 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.532046 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.532435 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.532537 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:12.535740 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:12.535753 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:12.535756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:12.535797 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:12.538301 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.538883 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.538886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.538888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.539016 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.539350 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.539353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.539355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.539639 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:12.539767 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:12.540550 (+ 783us) server_negotiation.cc:299] Negotiation successful
0618 22:33:12.540774 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:33:12.542157 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:12.524836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39547 (local address 127.18.150.131:46685)
0618 22:33:12.525593 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:12.525634 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:12.527454 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:12.528196 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:12.528208 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:12.528914 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:12.529609 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:12.529635 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.530689 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.530693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:12.531905 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:12.531915 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.532620 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.532628 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:12.532839 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:12.533413 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:12.533434 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:12.535521 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:12.538405 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:12.538411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:12.538423 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:12.538802 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:12.539092 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:12.539095 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:12.539097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:12.539267 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:12.539753 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:12.539761 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:12.540152 (+ 391us) client_negotiation.cc:769] Sending connection context
0618 22:33:12.540449 (+ 297us) client_negotiation.cc:240] Negotiation successful
0618 22:33:12.540925 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":277,"threads_started":1}
I20260618 22:33:12.543406 23091 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39547
I20260618 22:33:12.543704 23091 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:12.544375 23091 heartbeater.cc:511] Master 127.18.150.190:39547 requested a full tablet report, sending...
I20260618 22:33:12.545758 22635 ts_manager.cc:194] Registered new tserver with Master: b5066a5b92154ef9b6df2ea287a5bd88 (127.18.150.131:44987)
I20260618 22:33:12.546578 22635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:46685
I20260618 22:33:12.554006 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:33:12 dist-test-slave-h7jz krb5kdc[22590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:33:12 dist-test-slave-h7jz krb5kdc[22590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:33:12.597179 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:12.582943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45102 (local address 127.18.150.129:45459)
0618 22:33:12.583467 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:12.583475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:12.583507 (+ 32us) server_negotiation.cc:407] Connection header received
0618 22:33:12.583637 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:12.583644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:12.583859 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:12.584002 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:12.585372 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.586386 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.587665 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.588184 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.588296 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:12.591866 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:12.591876 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:12.591891 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:12.591922 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:12.594554 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.595231 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.595249 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.595251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.595363 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.595705 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.595709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.595711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.596036 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:12.596311 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:12.596580 (+ 269us) server_negotiation.cc:299] Negotiation successful
0618 22:33:12.596845 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":263,"threads_started":1}
Jun 18 22:33:12 dist-test-slave-h7jz krb5kdc[22590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:12.614393 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:12.601127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50918 (local address 127.18.150.190:39547)
0618 22:33:12.601340 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:12.601346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:12.601535 (+ 189us) server_negotiation.cc:407] Connection header received
0618 22:33:12.601594 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:12.601598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:12.601663 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:12.601743 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:12.602909 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.603694 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.604896 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.605307 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.605428 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:12.609173 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:12.609178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:12.609182 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:12.609220 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:12.611813 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.612437 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.612442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.612444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.612618 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.612936 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.612940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.612942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.613308 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:12.613530 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:12.613865 (+ 335us) server_negotiation.cc:299] Negotiation successful
0618 22:33:12.614082 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260618 22:33:12.614919 22635 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50918
I20260618 22:33:12.615907 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 22696
I20260618 22:33:12.626900 22824 heartbeater.cc:503] Master 127.18.150.190:39547 was elected leader, sending a full tablet report...
I20260618 22:33:12.628192 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:12.616727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.18.150.129:45459)
0618 22:33:12.616890 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:12.616897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:12.617142 (+ 245us) server_negotiation.cc:407] Connection header received
0618 22:33:12.617292 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:12.617296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:12.617382 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:12.617460 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:12.618675 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.619360 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.620532 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.620923 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.621028 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:12.623241 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:12.623246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:12.623250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:12.623282 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:12.625701 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.626366 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.626372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.626375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.626496 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.626842 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.626846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.626848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.627157 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:12.627363 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:12.627668 (+ 305us) server_negotiation.cc:299] Negotiation successful
0618 22:33:12.627871 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:33:12.628731 22819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:12.707901 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 22696
I20260618 22:33:12.720163 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 22830
Jun 18 22:33:12 dist-test-slave-h7jz krb5kdc[22590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:33:12.735141 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:12.721013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51776 (local address 127.18.150.130:45527)
0618 22:33:12.721534 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:12.721541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:12.721567 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:33:12.721625 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:12.721632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:12.721836 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:12.722007 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:12.723102 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.724132 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.725437 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.726017 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.726123 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:12.729846 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:12.729857 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:12.729872 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:12.729910 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:12.732457 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.733090 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.733097 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.733100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.733218 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.733549 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.733554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.733557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.733871 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:12.734174 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:12.734595 (+ 421us) server_negotiation.cc:299] Negotiation successful
0618 22:33:12.734836 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":243,"threads_started":1}
I20260618 22:33:12.736326 22953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:12.817929 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 22830
I20260618 22:33:12.829797 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 22963
Jun 18 22:33:12 dist-test-slave-h7jz krb5kdc[22590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:33:12.844695 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:12.830719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.18.150.131:44987)
0618 22:33:12.831302 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:12.831310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:12.831336 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:33:12.831391 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:12.831397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:12.831591 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:12.831747 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:12.832922 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.833932 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.835207 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:12.835753 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:12.835861 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:12.839691 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:12.839701 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:12.839716 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:12.839748 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:12.842204 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.842778 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.842786 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.842788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.842901 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:12.843229 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:12.843234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:12.843238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:12.843560 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:12.843842 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:12.844109 (+ 267us) server_negotiation.cc:299] Negotiation successful
0618 22:33:12.844372 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":298,"threads_started":1}
I20260618 22:33:12.845863 23086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:12.925797 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 22963
I20260618 22:33:12.937863 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 22603
I20260618 22:33:12.939108 22665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:13.037161 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 22603
2026-06-18T22:33:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3556 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:33:13 dist-test-slave-h7jz krb5kdc[23110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:33:13 dist-test-slave-h7jz krb5kdc[23110](info): set up 2 sockets
Jun 18 22:33:13 dist-test-slave-h7jz krb5kdc[23110](info): commencing operation
krb5kdc: starting...
W20260618 22:33:14.018173 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:33:14 dist-test-slave-h7jz krb5kdc[23110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:33:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:33:14.197494 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:32871
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:32871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:14.500741 23123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:14.501039 23123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:14.501186 23123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:14.508262 23123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:14.508389 23123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:14.508467 23123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:14.508543 23123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:14.508643 23123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:14.518491 23123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:32871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:32871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:14.519774 23123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:14.521086 23123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:14.531983 23128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:14.532104 23129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:14.532011 23131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:14.533195 23123 server_base.cc:1034] running on GCE node
I20260618 22:33:14.533986 23123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:14.535331 23123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:14.536527 23123 hybrid_clock.cc:648] HybridClock initialized: now 1781821994536466 us; error 52 us; skew 500 ppm
I20260618 22:33:14.536880 23123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:14 dist-test-slave-h7jz krb5kdc[23110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821994, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:14.544821 23123 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:33:14.546094 23123 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:14.546192 23123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:14.546499 23123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:14.549439 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "8b172ad27b55496ba1678c268df8e3fd"
format_stamp: "Formatted at 2026-06-18 22:33:14 on dist-test-slave-h7jz"
server_key: "01a6a84726d55596e368f22d98e47722"
server_key_iv: "9af190ce91af6c86873ab351498130c4"
server_key_version: "encryptionkey@0"
I20260618 22:33:14.550253 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "8b172ad27b55496ba1678c268df8e3fd"
format_stamp: "Formatted at 2026-06-18 22:33:14 on dist-test-slave-h7jz"
server_key: "01a6a84726d55596e368f22d98e47722"
server_key_iv: "9af190ce91af6c86873ab351498130c4"
server_key_version: "encryptionkey@0"
I20260618 22:33:14.555405 23123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260618 22:33:14.558947 23137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:14.560447 23123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:33:14.560626 23123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "8b172ad27b55496ba1678c268df8e3fd"
format_stamp: "Formatted at 2026-06-18 22:33:14 on dist-test-slave-h7jz"
server_key: "01a6a84726d55596e368f22d98e47722"
server_key_iv: "9af190ce91af6c86873ab351498130c4"
server_key_version: "encryptionkey@0"
I20260618 22:33:14.560845 23123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:14.579473 23123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:14.583724 23123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:14.583963 23123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:14.600867 23123 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:32871
I20260618 22:33:14.600885 23188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:32871 every 8 connection(s)
I20260618 22:33:14.601752 23123 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:14.603870 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 23123
I20260618 22:33:14.604128 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:14.605005 19034 external_mini_cluster.cc:1428] Setting key 2b8c826d0cff7fbcc942d807b2ce5d08
I20260618 22:33:14.606174 23189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 18 22:33:14 dist-test-slave-h7jz krb5kdc[23110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:14.617960 23189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd: Bootstrap starting.
I20260618 22:33:14.621134 23189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:14.622249 23189 log.cc:824] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:14.625245 23189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd: No bootstrap required, opened a new log
I20260618 22:33:14.625929 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:14.607198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38170 (local address 127.18.150.190:32871)
0618 22:33:14.607957 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:14.607969 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:14.608008 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:33:14.609500 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:14.609522 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:14.609913 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:14.610325 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:14.611719 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:14.613006 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:14.614238 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:14.614919 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:14.615093 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:14.619052 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:14.619059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:14.619073 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:14.619108 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:14.621873 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:14.622704 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:14.622710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:14.622716 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:14.622941 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:14.623334 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:14.623339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:14.623342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:14.623869 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:14.624075 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:14.624513 (+ 438us) server_negotiation.cc:299] Negotiation successful
0618 22:33:14.624827 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":356,"threads_started":1}
I20260618 22:33:14.632306 23189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b172ad27b55496ba1678c268df8e3fd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 32871 } }
I20260618 22:33:14.632635 23189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:14.632720 23189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b172ad27b55496ba1678c268df8e3fd, State: Initialized, Role: FOLLOWER
I20260618 22:33:14.633231 23189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b172ad27b55496ba1678c268df8e3fd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 32871 } }
I20260618 22:33:14.633425 23189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:14.633553 23189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:14.633688 23189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:14.635696 23189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b172ad27b55496ba1678c268df8e3fd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 32871 } }
I20260618 22:33:14.636139 23189 leader_election.cc:302] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b172ad27b55496ba1678c268df8e3fd; no voters:
I20260618 22:33:14.636646 23189 leader_election.cc:288] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:14.636776 23194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:14.637337 23194 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [term 1 LEADER]: Becoming Leader. State: Replica: 8b172ad27b55496ba1678c268df8e3fd, State: Running, Role: LEADER
I20260618 22:33:14.637807 23194 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b172ad27b55496ba1678c268df8e3fd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 32871 } }
I20260618 22:33:14.638561 23189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:14.640239 23196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b172ad27b55496ba1678c268df8e3fd. Latest consensus state: current_term: 1 leader_uuid: "8b172ad27b55496ba1678c268df8e3fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b172ad27b55496ba1678c268df8e3fd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 32871 } } }
I20260618 22:33:14.640578 23196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:14.640802 23195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b172ad27b55496ba1678c268df8e3fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b172ad27b55496ba1678c268df8e3fd" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 32871 } } }
I20260618 22:33:14.641089 23195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:14.641449 23199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:14.647013 23199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:14.657619 23199 catalog_manager.cc:1347] Generated new cluster ID: 5fd201e964e041eb97fe03e405a1c663
I20260618 22:33:14.657747 23199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:14.673575 23199 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:14.674831 23199 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:14.683495 23199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd: Generated new TSK 0
I20260618 22:33:14.684340 23199 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:33:14.741019 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:32871
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:15.074179 23215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:15.074432 23215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:15.074561 23215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:15.081640 23215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:15.081743 23215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:15.081898 23215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:33:15.091624 23215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:32871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:15.092867 23215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:15.094233 23215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:15.106271 23220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:15.106200 23223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:15.106223 23221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:15.106843 23215 server_base.cc:1034] running on GCE node
I20260618 22:33:15.107455 23215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:15.108400 23215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:15.109606 23215 hybrid_clock.cc:648] HybridClock initialized: now 1781821995109555 us; error 46 us; skew 500 ppm
I20260618 22:33:15.109921 23215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:15 dist-test-slave-h7jz krb5kdc[23110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821995, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:15.112972 23215 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:33:15.114107 23215 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:15.114212 23215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:15.114503 23215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:15.117388 23215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "5c8ab28a4b42469f9a733fb44d70cd76"
format_stamp: "Formatted at 2026-06-18 22:33:15 on dist-test-slave-h7jz"
server_key: "6179900a85e7dbfbacc9127513bcdbb0"
server_key_iv: "e6655a7e31090f9565ca29d4db27643d"
server_key_version: "encryptionkey@0"
I20260618 22:33:15.118203 23215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "5c8ab28a4b42469f9a733fb44d70cd76"
format_stamp: "Formatted at 2026-06-18 22:33:15 on dist-test-slave-h7jz"
server_key: "6179900a85e7dbfbacc9127513bcdbb0"
server_key_iv: "e6655a7e31090f9565ca29d4db27643d"
server_key_version: "encryptionkey@0"
I20260618 22:33:15.123142 23215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:33:15.126543 23229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:15.127882 23215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:33:15.128028 23215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "5c8ab28a4b42469f9a733fb44d70cd76"
format_stamp: "Formatted at 2026-06-18 22:33:15 on dist-test-slave-h7jz"
server_key: "6179900a85e7dbfbacc9127513bcdbb0"
server_key_iv: "e6655a7e31090f9565ca29d4db27643d"
server_key_version: "encryptionkey@0"
I20260618 22:33:15.128227 23215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:15.146332 23215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:15.150444 23215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:15.150704 23215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:15.151646 23215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:15.153030 23215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:15.153123 23215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:15.153272 23215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:15.153409 23215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:15.182377 23215 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:42701
I20260618 22:33:15.182528 23341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:42701 every 8 connection(s)
I20260618 22:33:15.183234 23215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:33:15.191802 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 23215
I20260618 22:33:15.192031 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:33:15.192663 19034 external_mini_cluster.cc:1428] Setting key 4b53ba20afcdf1d186e3385f3996f19a
Jun 18 22:33:15 dist-test-slave-h7jz krb5kdc[23110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821995, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:15.203037 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:15.185735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:53955 (local address 127.18.150.190:32871)
0618 22:33:15.186188 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:15.186196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:15.187322 (+ 1126us) server_negotiation.cc:407] Connection header received
0618 22:33:15.189078 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:15.189083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:15.189152 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:15.189257 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:15.190804 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:15.191688 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:15.193335 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:15.193832 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:15.194067 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:15.197686 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:15.197691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:15.197694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:15.197726 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:15.200086 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:15.200794 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:15.200798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:15.200800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:15.200911 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:15.201314 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:15.201318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:15.201320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:15.201617 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:15.201744 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:15.202466 (+ 722us) server_negotiation.cc:299] Negotiation successful
0618 22:33:15.202692 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":328,"thread_start_us":230,"threads_started":1}
I20260618 22:33:15.204001 23345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:15.186392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:32871 (local address 127.18.150.129:53955)
0618 22:33:15.187152 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:15.187197 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:15.188807 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:15.189442 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:15.189450 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:15.190052 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:15.190645 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:15.190664 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:15.191824 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:15.191828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:15.193174 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:15.193190 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:15.194063 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:15.194069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:15.194259 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:15.194953 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:15.194982 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:15.197471 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:15.200227 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:15.200236 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:15.200255 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:15.200666 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:15.201017 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:15.201022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:15.201027 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:15.201202 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:15.201743 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:15.201752 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:15.202118 (+ 366us) client_negotiation.cc:769] Sending connection context
0618 22:33:15.202345 (+ 227us) client_negotiation.cc:240] Negotiation successful
0618 22:33:15.202865 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":326,"threads_started":1}
I20260618 22:33:15.205483 23342 heartbeater.cc:347] Connected to a master server at 127.18.150.190:32871
I20260618 22:33:15.205787 23342 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:15.206423 23342 heartbeater.cc:511] Master 127.18.150.190:32871 requested a full tablet report, sending...
I20260618 22:33:15.208619 23154 ts_manager.cc:194] Registered new tserver with Master: 5c8ab28a4b42469f9a733fb44d70cd76 (127.18.150.129:42701)
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
I20260618 22:33:15.210305 23154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:53955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:33:15.233074 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:32871
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:15.536032 23348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:15.536294 23348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:15.536394 23348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:15.543586 23348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:15.543718 23348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:15.543854 23348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:33:15.553412 23348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:32871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:15.554675 23348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:15.555905 23348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:15.567986 23353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:15.567919 23356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:15.567960 23354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:15.568686 23348 server_base.cc:1034] running on GCE node
I20260618 22:33:15.569239 23348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:15.570101 23348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:15.571278 23348 hybrid_clock.cc:648] HybridClock initialized: now 1781821995571221 us; error 52 us; skew 500 ppm
I20260618 22:33:15.571559 23348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:15 dist-test-slave-h7jz krb5kdc[23110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821995, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:15.574330 23348 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:33:15.575330 23348 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:15.575414 23348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:15.575695 23348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:15.578454 23348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "4866cc11337f4215987369ad8fea88a4"
format_stamp: "Formatted at 2026-06-18 22:33:15 on dist-test-slave-h7jz"
server_key: "e2cd834c3313eb3d4add34ea01b92eea"
server_key_iv: "b971bfde9b6077d05ae9c155ebff3415"
server_key_version: "encryptionkey@0"
I20260618 22:33:15.579212 23348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "4866cc11337f4215987369ad8fea88a4"
format_stamp: "Formatted at 2026-06-18 22:33:15 on dist-test-slave-h7jz"
server_key: "e2cd834c3313eb3d4add34ea01b92eea"
server_key_iv: "b971bfde9b6077d05ae9c155ebff3415"
server_key_version: "encryptionkey@0"
I20260618 22:33:15.583904 23348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:33:15.587109 23362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:15.588423 23348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:33:15.588548 23348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "4866cc11337f4215987369ad8fea88a4"
format_stamp: "Formatted at 2026-06-18 22:33:15 on dist-test-slave-h7jz"
server_key: "e2cd834c3313eb3d4add34ea01b92eea"
server_key_iv: "b971bfde9b6077d05ae9c155ebff3415"
server_key_version: "encryptionkey@0"
I20260618 22:33:15.588735 23348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:15.613423 23348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:15.617117 23348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:15.617372 23348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:15.618283 23348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:15.619532 23348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:15.619614 23348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:15.619710 23348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:15.619776 23348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:15.647362 23348 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:36071
I20260618 22:33:15.647414 23474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:36071 every 8 connection(s)
I20260618 22:33:15.648130 23348 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:33:15.649993 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 23348
I20260618 22:33:15.650161 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:33:15.650636 19034 external_mini_cluster.cc:1428] Setting key c8e7a9661939c11760f71ec02b9304c0
Jun 18 22:33:15 dist-test-slave-h7jz krb5kdc[23110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821995, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:15.668929 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:15.650357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:45273 (local address 127.18.150.190:32871)
0618 22:33:15.650572 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:15.650579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:15.651831 (+ 1252us) server_negotiation.cc:407] Connection header received
0618 22:33:15.654036 (+ 2205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:15.654040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:15.654101 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:15.654191 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:15.656011 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:15.656706 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:15.658737 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:15.659340 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:15.659463 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:15.662918 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:15.662923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:15.662928 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:15.662962 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:15.666084 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:15.666764 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:15.666770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:15.666774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:15.666925 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:15.667279 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:15.667284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:15.667287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:15.667616 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:15.667757 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:15.668372 (+ 615us) server_negotiation.cc:299] Negotiation successful
0618 22:33:15.668585 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:33:15.669715 23477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:15.650928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:32871 (local address 127.18.150.130:45273)
0618 22:33:15.651699 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:15.651737 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:15.653731 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:15.654450 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:15.654462 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:15.655128 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:15.655860 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:15.655884 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:15.656841 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:15.656847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:15.658610 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:15.658623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:15.659535 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:15.659540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:15.659754 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:15.660415 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:15.660446 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:15.662708 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:15.666227 (+ 3519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:15.666233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:15.666245 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:15.666636 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:15.667017 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:15.667020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:15.667022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:15.667176 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:15.667724 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:15.667730 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:15.668028 (+ 298us) client_negotiation.cc:769] Sending connection context
0618 22:33:15.668233 (+ 205us) client_negotiation.cc:240] Negotiation successful
0618 22:33:15.668619 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":301,"threads_started":1}
I20260618 22:33:15.671062 23475 heartbeater.cc:347] Connected to a master server at 127.18.150.190:32871
I20260618 22:33:15.671358 23475 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
I20260618 22:33:15.671983 23475 heartbeater.cc:511] Master 127.18.150.190:32871 requested a full tablet report, sending...
I20260618 22:33:15.673409 23154 ts_manager.cc:194] Registered new tserver with Master: 4866cc11337f4215987369ad8fea88a4 (127.18.150.130:36071)
I20260618 22:33:15.674392 23154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:45273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:33:15.693130 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:32871
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:16.023144 23480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:16.023409 23480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:16.023502 23480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:16.030520 23480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:16.030628 23480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:16.030742 23480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:33:16.040376 23480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:32871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:16.041639 23480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:16.042908 23480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:16.054726 23488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:16.054841 23485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:16.054893 23486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:16.055748 23480 server_base.cc:1034] running on GCE node
I20260618 22:33:16.056318 23480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:16.057178 23480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:16.058360 23480 hybrid_clock.cc:648] HybridClock initialized: now 1781821996058318 us; error 36 us; skew 500 ppm
I20260618 22:33:16.058655 23480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:16 dist-test-slave-h7jz krb5kdc[23110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821996, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:16.061499 23480 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:33:16.062587 23480 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:16.062685 23480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:16.062918 23480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:16.065747 23480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "9858a22fbb07482a8e8ac3e44b2be0bc"
format_stamp: "Formatted at 2026-06-18 22:33:16 on dist-test-slave-h7jz"
server_key: "e67f4c832240ac1e3ebb361ca7909995"
server_key_iv: "dc91b8c9c3193134f75a6a093b798160"
server_key_version: "encryptionkey@0"
I20260618 22:33:16.066529 23480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "9858a22fbb07482a8e8ac3e44b2be0bc"
format_stamp: "Formatted at 2026-06-18 22:33:16 on dist-test-slave-h7jz"
server_key: "e67f4c832240ac1e3ebb361ca7909995"
server_key_iv: "dc91b8c9c3193134f75a6a093b798160"
server_key_version: "encryptionkey@0"
I20260618 22:33:16.071214 23480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260618 22:33:16.074536 23494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:16.075941 23480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:16.076097 23480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "9858a22fbb07482a8e8ac3e44b2be0bc"
format_stamp: "Formatted at 2026-06-18 22:33:16 on dist-test-slave-h7jz"
server_key: "e67f4c832240ac1e3ebb361ca7909995"
server_key_iv: "dc91b8c9c3193134f75a6a093b798160"
server_key_version: "encryptionkey@0"
I20260618 22:33:16.076288 23480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:16.090603 23480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:16.094466 23480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:16.094726 23480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:16.095605 23480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:16.096906 23480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:16.096997 23480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:16.097100 23480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:16.097167 23480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:16.126227 23480 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:35021
I20260618 22:33:16.126277 23606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:35021 every 8 connection(s)
I20260618 22:33:16.127022 23480 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:33:16.131114 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 23480
I20260618 22:33:16.131309 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:33:16.131861 19034 external_mini_cluster.cc:1428] Setting key cc5566a9086a863414911c368dbab3bf
Jun 18 22:33:16 dist-test-slave-h7jz krb5kdc[23110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821996, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:16.146926 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.129300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:54229 (local address 127.18.150.190:32871)
0618 22:33:16.129493 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:16.129502 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:16.130723 (+ 1221us) server_negotiation.cc:407] Connection header received
0618 22:33:16.133018 (+ 2295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:16.133022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:16.133139 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:16.133245 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:16.135123 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.135841 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.137364 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.137774 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.137868 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.141167 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:16.141172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:16.141175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:16.141206 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:16.143859 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:16.144582 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:16.144585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:16.144588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:16.144713 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:16.145067 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:16.145071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:16.145074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:16.145398 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:16.145534 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:16.146375 (+ 841us) server_negotiation.cc:299] Negotiation successful
0618 22:33:16.146588 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:33:16.147888 23609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.129838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:32871 (local address 127.18.150.131:54229)
0618 22:33:16.130536 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.130606 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:16.132738 (+ 2132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:16.133516 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:16.133528 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:16.134193 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:16.134932 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.134956 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.135983 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.135986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.137230 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.137241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.137967 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.137974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.138171 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.138798 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:16.138822 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:16.140960 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:16.144025 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:16.144033 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:16.144060 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:16.144483 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:16.144795 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:16.144800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:16.144802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:16.144966 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:16.145524 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:16.145530 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:16.145983 (+ 453us) client_negotiation.cc:769] Sending connection context
0618 22:33:16.146256 (+ 273us) client_negotiation.cc:240] Negotiation successful
0618 22:33:16.146777 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":283,"threads_started":1}
I20260618 22:33:16.149227 23607 heartbeater.cc:347] Connected to a master server at 127.18.150.190:32871
I20260618 22:33:16.149571 23607 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:16.150317 23607 heartbeater.cc:511] Master 127.18.150.190:32871 requested a full tablet report, sending...
I20260618 22:33:16.151661 23154 ts_manager.cc:194] Registered new tserver with Master: 9858a22fbb07482a8e8ac3e44b2be0bc (127.18.150.131:35021)
I20260618 22:33:16.152469 23154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:54229
I20260618 22:33:16.159288 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:33:16.174943 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.163478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38178 (local address 127.18.150.190:32871)
0618 22:33:16.163652 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:16.163659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:16.164105 (+ 446us) server_negotiation.cc:407] Connection header received
0618 22:33:16.164281 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:16.164284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:16.164358 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:16.164438 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:16.165772 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.166549 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.167775 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.168207 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.168295 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.170447 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:16.170452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:16.170455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:16.170487 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:16.172775 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:16.173322 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:16.173326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:16.173328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:16.173468 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:16.173737 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:16.173740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:16.173742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:16.174085 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:16.174267 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:16.174504 (+ 237us) server_negotiation.cc:299] Negotiation successful
0618 22:33:16.174662 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:33:16.178704 23154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:33:16.185479 23154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:33:16.206614 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.196987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44976 (local address 127.18.150.129:42701)
0618 22:33:16.198792 (+ 1805us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:16.198800 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:16.198984 (+ 184us) server_negotiation.cc:407] Connection header received
0618 22:33:16.199254 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:16.199260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:16.199455 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:16.199630 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:16.200195 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.201737 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.204476 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.205532 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.205711 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.205886 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:16.206153 (+ 267us) server_negotiation.cc:299] Negotiation successful
0618 22:33:16.206311 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1655,"thread_start_us":1977,"threads_started":1}
I20260618 22:33:16.206720 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.196959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:42701 (local address 127.0.0.1:44976)
0618 22:33:16.198829 (+ 1870us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.198880 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:16.199068 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:16.199650 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:16.199657 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:16.199685 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:16.199985 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.199997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.201881 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.201886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.204307 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.204321 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.205727 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.205733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.205916 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.205994 (+ 78us) client_negotiation.cc:769] Sending connection context
0618 22:33:16.206104 (+ 110us) client_negotiation.cc:240] Negotiation successful
0618 22:33:16.206283 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1602,"mutex_wait_us":83,"thread_start_us":2375,"threads_started":1}
I20260618 22:33:16.207187 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.197327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:36071 (local address 127.0.0.1:48548)
0618 22:33:16.199551 (+ 2224us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.199567 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:16.199680 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:16.200522 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:16.200528 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:16.200552 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:16.200959 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.200970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.202785 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.202793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.204626 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.204636 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.206409 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.206420 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.206573 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.206607 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:33:16.206670 (+ 63us) client_negotiation.cc:240] Negotiation successful
0618 22:33:16.206807 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2120,"mutex_wait_us":24,"thread_start_us":1095,"threads_started":1}
I20260618 22:33:16.208165 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.197611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48548 (local address 127.18.150.130:36071)
0618 22:33:16.198392 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:16.198401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:16.199788 (+ 1387us) server_negotiation.cc:407] Connection header received
0618 22:33:16.199887 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:16.199894 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:16.200119 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:16.200292 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:16.201110 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.202608 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.204755 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.206172 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.207428 (+ 1256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.207615 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:16.207681 (+ 66us) server_negotiation.cc:299] Negotiation successful
0618 22:33:16.207847 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":463,"threads_started":1}
I20260618 22:33:16.210034 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.200383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:35021 (local address 127.0.0.1:34950)
0618 22:33:16.201429 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.201446 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:16.201548 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:16.202467 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:16.202471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:16.202492 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:16.202975 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.202987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.204873 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.204878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.206817 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.206829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.209237 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.209243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.209413 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.209454 (+ 41us) client_negotiation.cc:769] Sending connection context
0618 22:33:16.209544 (+ 90us) client_negotiation.cc:240] Negotiation successful
0618 22:33:16.209669 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"mutex_wait_us":43,"thread_start_us":818,"threads_started":1}
I20260618 22:33:16.210085 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.199999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.18.150.131:35021)
0618 22:33:16.202055 (+ 2056us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:16.202062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:16.202086 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:33:16.202155 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:16.202159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:16.202304 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:16.202454 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:16.203217 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.204745 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.206980 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.208334 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.209240 (+ 906us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.209397 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:16.209609 (+ 212us) server_negotiation.cc:299] Negotiation successful
0618 22:33:16.209762 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1909,"thread_start_us":1762,"threads_started":1}
I20260618 22:33:16.210875 23277 tablet_service.cc:1467] Processing CreateTablet for tablet 77f5aef29ce145aabe3dfa16205077f3 (DEFAULT_TABLE table=test-table [id=cfd1d8d830ec4581bbcb0a25940c6c7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:16.211372 23410 tablet_service.cc:1467] Processing CreateTablet for tablet 77f5aef29ce145aabe3dfa16205077f3 (DEFAULT_TABLE table=test-table [id=cfd1d8d830ec4581bbcb0a25940c6c7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:16.212213 23277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77f5aef29ce145aabe3dfa16205077f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:16.212756 23410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77f5aef29ce145aabe3dfa16205077f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:16.213786 23542 tablet_service.cc:1467] Processing CreateTablet for tablet 77f5aef29ce145aabe3dfa16205077f3 (DEFAULT_TABLE table=test-table [id=cfd1d8d830ec4581bbcb0a25940c6c7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:16.215065 23542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77f5aef29ce145aabe3dfa16205077f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:16.218209 23342 heartbeater.cc:503] Master 127.18.150.190:32871 was elected leader, sending a full tablet report...
I20260618 22:33:16.223677 23625 tablet_bootstrap.cc:492] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76: Bootstrap starting.
I20260618 22:33:16.227555 23625 tablet_bootstrap.cc:654] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:16.227794 23626 tablet_bootstrap.cc:492] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: Bootstrap starting.
I20260618 22:33:16.228945 23625 log.cc:824] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:16.231264 23626 tablet_bootstrap.cc:654] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:16.232134 23625 tablet_bootstrap.cc:492] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76: No bootstrap required, opened a new log
I20260618 22:33:16.232403 23625 ts_tablet_manager.cc:1397] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260618 22:33:16.232522 23626 log.cc:824] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:16.233353 23627 tablet_bootstrap.cc:492] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: Bootstrap starting.
I20260618 22:33:16.235249 23626 tablet_bootstrap.cc:492] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: No bootstrap required, opened a new log
I20260618 22:33:16.235559 23626 ts_tablet_manager.cc:1397] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260618 22:33:16.236701 23627 tablet_bootstrap.cc:654] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:16.237829 23627 log.cc:824] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:16.240780 23627 tablet_bootstrap.cc:492] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: No bootstrap required, opened a new log
I20260618 22:33:16.241044 23627 ts_tablet_manager.cc:1397] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260618 22:33:16.241876 23625 raft_consensus.cc:348] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 42701 } } peers { permanent_uuid: "4866cc11337f4215987369ad8fea88a4" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 36071 } } peers { permanent_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 35021 } }
I20260618 22:33:16.242340 23625 raft_consensus.cc:374] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:16.242437 23625 raft_consensus.cc:729] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c8ab28a4b42469f9a733fb44d70cd76, State: Initialized, Role: FOLLOWER
I20260618 22:33:16.242820 23626 raft_consensus.cc:348] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 42701 } } peers { permanent_uuid: "4866cc11337f4215987369ad8fea88a4" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 36071 } } peers { permanent_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 35021 } }
I20260618 22:33:16.243145 23626 raft_consensus.cc:374] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:16.243032 23625 consensus_queue.cc:260] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 42701 } } peers { permanent_uuid: "4866cc11337f4215987369ad8fea88a4" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 36071 } } peers { permanent_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 35021 } }
I20260618 22:33:16.243234 23626 raft_consensus.cc:729] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4866cc11337f4215987369ad8fea88a4, State: Initialized, Role: FOLLOWER
I20260618 22:33:16.243861 23626 consensus_queue.cc:260] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 42701 } } peers { permanent_uuid: "4866cc11337f4215987369ad8fea88a4" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 36071 } } peers { permanent_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 35021 } }
I20260618 22:33:16.244905 23475 heartbeater.cc:503] Master 127.18.150.190:32871 was elected leader, sending a full tablet report...
I20260618 22:33:16.245517 23626 ts_tablet_manager.cc:1428] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260618 22:33:16.248365 23625 ts_tablet_manager.cc:1428] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260618 22:33:16.250826 23627 raft_consensus.cc:348] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 42701 } } peers { permanent_uuid: "4866cc11337f4215987369ad8fea88a4" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 36071 } } peers { permanent_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 35021 } }
I20260618 22:33:16.251138 23627 raft_consensus.cc:374] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:16.251224 23627 raft_consensus.cc:729] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9858a22fbb07482a8e8ac3e44b2be0bc, State: Initialized, Role: FOLLOWER
I20260618 22:33:16.251703 23627 consensus_queue.cc:260] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 42701 } } peers { permanent_uuid: "4866cc11337f4215987369ad8fea88a4" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 36071 } } peers { permanent_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 35021 } }
I20260618 22:33:16.252583 23607 heartbeater.cc:503] Master 127.18.150.190:32871 was elected leader, sending a full tablet report...
I20260618 22:33:16.253139 23627 ts_tablet_manager.cc:1428] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260618 22:33:16.378434 23608 tablet.cc:2369] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:33:16.400302 23476 tablet.cc:2369] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:33:16.431728 23631 raft_consensus.cc:482] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:33:16.432030 23631 raft_consensus.cc:504] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 42701 } } peers { permanent_uuid: "4866cc11337f4215987369ad8fea88a4" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 36071 } } peers { permanent_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 35021 } }
I20260618 22:33:16.434022 23631 leader_election.cc:288] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4866cc11337f4215987369ad8fea88a4 (127.18.150.130:36071), 9858a22fbb07482a8e8ac3e44b2be0bc (127.18.150.131:35021)
W20260618 22:33:16.435506 23343 tablet.cc:2369] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:33:16.440955 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.434448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:40615 (local address 127.18.150.130:36071)
0618 22:33:16.434628 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:16.434634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:16.435815 (+ 1181us) server_negotiation.cc:407] Connection header received
0618 22:33:16.436085 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:16.436090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:16.436169 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:16.436267 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:16.436866 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.437542 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.439349 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.440115 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.440251 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.440324 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:16.440585 (+ 261us) server_negotiation.cc:299] Negotiation successful
0618 22:33:16.440708 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:33:16.441049 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.434417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:53433 (local address 127.18.150.131:35021)
0618 22:33:16.434606 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:16.434613 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:16.435745 (+ 1132us) server_negotiation.cc:407] Connection header received
0618 22:33:16.436014 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:16.436020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:16.436107 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:16.436183 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:16.436702 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.437398 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.439339 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.440222 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.440358 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.440426 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:16.440707 (+ 281us) server_negotiation.cc:299] Negotiation successful
0618 22:33:16.440822 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:33:16.441120 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.434470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:36071 (local address 127.18.150.129:40615)
0618 22:33:16.435713 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.435730 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:16.435906 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:16.436414 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:16.436417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:16.436435 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:16.436730 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.436737 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.437674 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.437678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.439200 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.439212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.440305 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.440311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.440443 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.440466 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:33:16.440556 (+ 90us) client_negotiation.cc:240] Negotiation successful
0618 22:33:16.440678 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"mutex_wait_us":156,"spinlock_wait_cycles":25856,"thread_start_us":373,"threads_started":1}
I20260618 22:33:16.441210 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.434463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:35021 (local address 127.18.150.129:53433)
0618 22:33:16.435632 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.435651 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:16.435859 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:16.436210 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:16.436214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:16.436237 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:16.436579 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.436588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.437518 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.437524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.439200 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:16.439213 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.440422 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.440428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:16.440577 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.440599 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:33:16.440673 (+ 74us) client_negotiation.cc:240] Negotiation successful
0618 22:33:16.440789 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"mutex_wait_us":36,"thread_start_us":487,"threads_started":1}
I20260618 22:33:16.442095 23562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f5aef29ce145aabe3dfa16205077f3" candidate_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" is_pre_election: true
I20260618 22:33:16.442309 23430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f5aef29ce145aabe3dfa16205077f3" candidate_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4866cc11337f4215987369ad8fea88a4" is_pre_election: true
I20260618 22:33:16.442548 23562 raft_consensus.cc:2454] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c8ab28a4b42469f9a733fb44d70cd76 in term 0.
I20260618 22:33:16.442771 23430 raft_consensus.cc:2454] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c8ab28a4b42469f9a733fb44d70cd76 in term 0.
I20260618 22:33:16.443224 23233 leader_election.cc:302] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c8ab28a4b42469f9a733fb44d70cd76, 9858a22fbb07482a8e8ac3e44b2be0bc; no voters:
I20260618 22:33:16.443572 23631 raft_consensus.cc:2781] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:33:16.443703 23631 raft_consensus.cc:482] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:33:16.443799 23631 raft_consensus.cc:3037] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:16.445835 23631 raft_consensus.cc:504] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 42701 } } peers { permanent_uuid: "4866cc11337f4215987369ad8fea88a4" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 36071 } } peers { permanent_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 35021 } }
I20260618 22:33:16.446439 23631 leader_election.cc:288] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [CANDIDATE]: Term 1 election: Requested vote from peers 4866cc11337f4215987369ad8fea88a4 (127.18.150.130:36071), 9858a22fbb07482a8e8ac3e44b2be0bc (127.18.150.131:35021)
I20260618 22:33:16.446852 23562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f5aef29ce145aabe3dfa16205077f3" candidate_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc"
I20260618 22:33:16.446866 23430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77f5aef29ce145aabe3dfa16205077f3" candidate_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4866cc11337f4215987369ad8fea88a4"
I20260618 22:33:16.447021 23562 raft_consensus.cc:3037] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:16.447023 23430 raft_consensus.cc:3037] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:16.449035 23562 raft_consensus.cc:2454] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c8ab28a4b42469f9a733fb44d70cd76 in term 1.
I20260618 22:33:16.449080 23430 raft_consensus.cc:2454] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c8ab28a4b42469f9a733fb44d70cd76 in term 1.
I20260618 22:33:16.449370 23233 leader_election.cc:302] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c8ab28a4b42469f9a733fb44d70cd76, 9858a22fbb07482a8e8ac3e44b2be0bc; no voters:
I20260618 22:33:16.449644 23631 raft_consensus.cc:2781] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:16.450107 23631 raft_consensus.cc:686] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 1 LEADER]: Becoming Leader. State: Replica: 5c8ab28a4b42469f9a733fb44d70cd76, State: Running, Role: LEADER
I20260618 22:33:16.450562 23631 consensus_queue.cc:237] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 42701 } } peers { permanent_uuid: "4866cc11337f4215987369ad8fea88a4" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 36071 } } peers { permanent_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 35021 } }
I20260618 22:33:16.454905 23153 catalog_manager.cc:5495] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c8ab28a4b42469f9a733fb44d70cd76 (127.18.150.129). New cstate: current_term: 1 leader_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c8ab28a4b42469f9a733fb44d70cd76" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 42701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4866cc11337f4215987369ad8fea88a4" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 36071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 35021 } health_report { overall_health: UNKNOWN } } }
I20260618 22:33:16.501523 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.496769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.18.150.129:42701)
0618 22:33:16.496936 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:16.496940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:16.497102 (+ 162us) server_negotiation.cc:407] Connection header received
0618 22:33:16.497241 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:16.497244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:16.497322 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:16.497445 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:33:16.497723 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.498433 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.499657 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.500069 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.500214 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.500614 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:33:16.501099 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:33:16.501221 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:16.501262 (+ 41us) server_negotiation.cc:299] Negotiation successful
0618 22:33:16.501325 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:33:16.509572 23430 raft_consensus.cc:1261] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4 [term 1 FOLLOWER]: Refusing update from remote peer 5c8ab28a4b42469f9a733fb44d70cd76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:16.509572 23562 raft_consensus.cc:1261] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc [term 1 FOLLOWER]: Refusing update from remote peer 5c8ab28a4b42469f9a733fb44d70cd76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:16.510589 23636 consensus_queue.cc:1035] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4866cc11337f4215987369ad8fea88a4" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 36071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:16.510906 23631 consensus_queue.cc:1035] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9858a22fbb07482a8e8ac3e44b2be0bc" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 35021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:16.535413 23153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:16.535652 23153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:16.539074 23153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b172ad27b55496ba1678c268df8e3fd: Sending DeleteTablet for 3 replicas of tablet 77f5aef29ce145aabe3dfa16205077f3
I20260618 22:33:16.540176 23410 tablet_service.cc:1514] Processing DeleteTablet for tablet 77f5aef29ce145aabe3dfa16205077f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:16 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:48548
I20260618 22:33:16.540264 23542 tablet_service.cc:1514] Processing DeleteTablet for tablet 77f5aef29ce145aabe3dfa16205077f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:16 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:34950
I20260618 22:33:16.540304 23277 tablet_service.cc:1514] Processing DeleteTablet for tablet 77f5aef29ce145aabe3dfa16205077f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:16 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:44976
I20260618 22:33:16.542169 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 23215
I20260618 22:33:16.551256 23651 tablet_replica.cc:330] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: stopping tablet replica
I20260618 22:33:16.551873 23651 raft_consensus.cc:2229] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:16.552452 23651 raft_consensus.cc:2258] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:16.553037 23650 tablet_replica.cc:330] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76: stopping tablet replica
I20260618 22:33:16.553990 23649 tablet_replica.cc:330] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: stopping tablet replica
I20260618 22:33:16.553984 23650 raft_consensus.cc:2229] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:33:16.554395 23649 raft_consensus.cc:2229] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:16.554903 23649 raft_consensus.cc:2258] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:16.556025 23651 ts_tablet_manager.cc:1905] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:16.556562 23650 raft_consensus.cc:2258] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:16.556924 23649 ts_tablet_manager.cc:1905] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 18 22:33:16 dist-test-slave-h7jz krb5kdc[23110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:33:16.563598 23649 ts_tablet_manager.cc:1918] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:16.563692 23651 ts_tablet_manager.cc:1918] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:16.563803 23649 log.cc:1196] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/77f5aef29ce145aabe3dfa16205077f3
I20260618 22:33:16.563884 23651 log.cc:1196] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/77f5aef29ce145aabe3dfa16205077f3
I20260618 22:33:16.564376 23649 ts_tablet_manager.cc:1939] T 77f5aef29ce145aabe3dfa16205077f3 P 4866cc11337f4215987369ad8fea88a4: Deleting consensus metadata
I20260618 22:33:16.564438 23651 ts_tablet_manager.cc:1939] T 77f5aef29ce145aabe3dfa16205077f3 P 9858a22fbb07482a8e8ac3e44b2be0bc: Deleting consensus metadata
I20260618 22:33:16.566303 23141 catalog_manager.cc:4841] TS 9858a22fbb07482a8e8ac3e44b2be0bc (127.18.150.131:35021): tablet 77f5aef29ce145aabe3dfa16205077f3 (table test-table [id=cfd1d8d830ec4581bbcb0a25940c6c7b]) successfully deleted
I20260618 22:33:16.566368 23138 catalog_manager.cc:4841] TS 4866cc11337f4215987369ad8fea88a4 (127.18.150.130:36071): tablet 77f5aef29ce145aabe3dfa16205077f3 (table test-table [id=cfd1d8d830ec4581bbcb0a25940c6c7b]) successfully deleted
I20260618 22:33:16.568460 23650 ts_tablet_manager.cc:1905] T 77f5aef29ce145aabe3dfa16205077f3 P 5c8ab28a4b42469f9a733fb44d70cd76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:16.572320 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.543530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45004 (local address 127.18.150.129:42701)
0618 22:33:16.543707 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:16.543714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:16.553264 (+ 9550us) server_negotiation.cc:407] Connection header received
0618 22:33:16.553506 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:16.553512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:16.553590 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:16.553686 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:16.555358 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.556508 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.558191 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.558813 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.558911 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.564356 (+ 5445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:16.564375 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:16.564393 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:16.564442 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:16.568801 (+ 4359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:16.569644 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:16.569653 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:16.569656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:16.569854 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:16.570377 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:16.570382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:16.570386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:16.570846 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:16.571121 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:16.571670 (+ 549us) server_negotiation.cc:299] Negotiation successful
0618 22:33:16.571900 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:33:16.573760 23337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:16.670548 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 23215
I20260618 22:33:16.683894 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 23348
W20260618 22:33:16.683859 23141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.150.129:42701 (error 108)
I20260618 22:33:16.685359 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.684998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:42701 (local address 127.0.0.1:45014)
0618 22:33:16.685143 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.685230 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:42701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260618 22:33:16.685580 23141 catalog_manager.cc:4584] TS 5c8ab28a4b42469f9a733fb44d70cd76 (127.18.150.129:42701): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77f5aef29ce145aabe3dfa16205077f3: Network error: Client connection negotiation failed: client connection to 127.18.150.129:42701: connect: Connection refused (error 111)
Jun 18 22:33:16 dist-test-slave-h7jz krb5kdc[23110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:33:16.698150 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.684606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.18.150.130:36071)
0618 22:33:16.684778 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:16.684784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:16.684870 (+ 86us) server_negotiation.cc:407] Connection header received
0618 22:33:16.685044 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:16.685048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:16.685108 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:16.685189 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:16.686547 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.687251 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.688480 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.688854 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.688956 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.692826 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:16.692844 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:16.692864 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:16.692911 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:16.695551 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:16.696208 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:16.696215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:16.696217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:16.696348 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:16.696692 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:16.696696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:16.696698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:16.697057 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:16.697324 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:16.697613 (+ 289us) server_negotiation.cc:299] Negotiation successful
0618 22:33:16.697814 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:33:16.698755 23470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:16.739090 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.738646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:42701 (local address 127.0.0.1:45026)
0618 22:33:16.738837 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.738938 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:42701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260618 22:33:16.740015 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.739708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:42701 (local address 127.0.0.1:45036)
0618 22:33:16.739819 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.739904 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:42701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260618 22:33:16.786652 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 23348
I20260618 22:33:16.799073 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 23480
I20260618 22:33:16.800493 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.800050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:42701 (local address 127.0.0.1:45050)
0618 22:33:16.800214 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.800301 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:42701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260618 22:33:16.801456 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.801131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:42701 (local address 127.0.0.1:45052)
0618 22:33:16.801279 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.801349 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:42701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
Jun 18 22:33:16 dist-test-slave-h7jz krb5kdc[23110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:33:16.813263 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.799796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34962 (local address 127.18.150.131:35021)
0618 22:33:16.799959 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:16.799963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:16.800025 (+ 62us) server_negotiation.cc:407] Connection header received
0618 22:33:16.800253 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:16.800258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:16.800345 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:16.800443 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:16.801759 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.802486 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.803780 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:16.804161 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:16.804269 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:16.808289 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:16.808304 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:16.808322 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:16.808356 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:16.810827 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:16.811452 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:16.811459 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:16.811461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:16.811583 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:16.811984 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:16.811987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:16.811989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:16.812346 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:16.812547 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:16.812801 (+ 254us) server_negotiation.cc:299] Negotiation successful
0618 22:33:16.812953 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:33:16.813907 23602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:16.882167 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.881571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:42701 (local address 127.0.0.1:45064)
0618 22:33:16.881803 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.881999 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:42701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260618 22:33:16.883157 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:16.882799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:42701 (local address 127.0.0.1:45080)
0618 22:33:16.882975 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:16.883049 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:42701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260618 22:33:16.900573 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 23480
I20260618 22:33:16.913240 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 23123
I20260618 22:33:16.914330 23184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:17.021735 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 23123
2026-06-18T22:33:17Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3985 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-18T22:33:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:17Z Disabled control of system clock
I20260618 22:33:17.078334 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:40287
--webserver_interface=127.18.150.190
--webserver_port=0
--builtin_ntp_servers=127.18.150.148:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:40287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260618 22:33:17.380468 23661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:17.380743 23661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:17.380827 23661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:17.388262 23661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:17.388366 23661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:17.388410 23661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:17.388463 23661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:17.388504 23661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:17.397789 23661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:40287
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:40287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:17.399029 23661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:17.400662 23661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:17.411415 23666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:17.411476 23669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:17.411451 23667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:17.412241 23661 server_base.cc:1034] running on GCE node
I20260618 22:33:17.412986 23661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:17.414338 23661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:17.415530 23661 hybrid_clock.cc:648] HybridClock initialized: now 1781821997415482 us; error 44 us; skew 500 ppm
I20260618 22:33:17.415839 23661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:33:17.417824 23661 webserver.cc:460] Webserver started at http://127.18.150.190:42877/ using document root <none> and password file <none>
I20260618 22:33:17.418494 23661 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:17.418588 23661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:17.418871 23661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:17.421517 23661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "3ec0714c8d524b5ab5922859329d37d6"
format_stamp: "Formatted at 2026-06-18 22:33:17 on dist-test-slave-h7jz"
server_key: "b5f2c5e5662d561cadfc1b80519f56c2"
server_key_iv: "9cdcd8e8af1ce40d779c8fcb36e81eb2"
server_key_version: "encryptionkey@0"
I20260618 22:33:17.422267 23661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "3ec0714c8d524b5ab5922859329d37d6"
format_stamp: "Formatted at 2026-06-18 22:33:17 on dist-test-slave-h7jz"
server_key: "b5f2c5e5662d561cadfc1b80519f56c2"
server_key_iv: "9cdcd8e8af1ce40d779c8fcb36e81eb2"
server_key_version: "encryptionkey@0"
I20260618 22:33:17.426937 23661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260618 22:33:17.430022 23675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:17.431273 23661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260618 22:33:17.431414 23661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "3ec0714c8d524b5ab5922859329d37d6"
format_stamp: "Formatted at 2026-06-18 22:33:17 on dist-test-slave-h7jz"
server_key: "b5f2c5e5662d561cadfc1b80519f56c2"
server_key_iv: "9cdcd8e8af1ce40d779c8fcb36e81eb2"
server_key_version: "encryptionkey@0"
I20260618 22:33:17.431587 23661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:17.436878 23661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:17.437565 23661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:17.437814 23661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:17.454109 23661 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:40287
I20260618 22:33:17.454130 23726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:40287 every 8 connection(s)
I20260618 22:33:17.455355 23661 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:17.458984 23727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:17.463420 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 23661
I20260618 22:33:17.463601 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:17.464399 19034 external_mini_cluster.cc:1428] Setting key 9fd8efcf4c077c3687d631aa7bb57ce8
I20260618 22:33:17.469825 23727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6: Bootstrap starting.
I20260618 22:33:17.472106 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:17.466298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49296 (local address 127.18.150.190:40287)
0618 22:33:17.467238 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:17.467259 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:17.467304 (+ 45us) server_negotiation.cc:407] Connection header received
0618 22:33:17.468901 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:17.468924 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:17.469317 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:17.469640 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:17.470611 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:17.470616 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:17.470629 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:17.470642 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:17.470698 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:17.470713 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:17.470898 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:17.471019 (+ 121us) server_negotiation.cc:299] Negotiation successful
0618 22:33:17.471286 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":317,"threads_started":1}
I20260618 22:33:17.473098 23727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:17.474148 23727 log.cc:824] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:17.476619 23727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6: No bootstrap required, opened a new log
I20260618 22:33:17.483651 23727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec0714c8d524b5ab5922859329d37d6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40287 } }
I20260618 22:33:17.483971 23727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:17.484062 23727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec0714c8d524b5ab5922859329d37d6, State: Initialized, Role: FOLLOWER
I20260618 22:33:17.484611 23727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec0714c8d524b5ab5922859329d37d6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40287 } }
I20260618 22:33:17.484810 23727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:17.484926 23727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:17.485082 23727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:17.487413 23727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec0714c8d524b5ab5922859329d37d6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40287 } }
I20260618 22:33:17.487897 23727 leader_election.cc:302] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ec0714c8d524b5ab5922859329d37d6; no voters:
I20260618 22:33:17.488436 23727 leader_election.cc:288] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:17.488551 23732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:17.489128 23732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [term 1 LEADER]: Becoming Leader. State: Replica: 3ec0714c8d524b5ab5922859329d37d6, State: Running, Role: LEADER
I20260618 22:33:17.489648 23732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec0714c8d524b5ab5922859329d37d6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40287 } }
I20260618 22:33:17.490419 23727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:17.492189 23734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ec0714c8d524b5ab5922859329d37d6. Latest consensus state: current_term: 1 leader_uuid: "3ec0714c8d524b5ab5922859329d37d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec0714c8d524b5ab5922859329d37d6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40287 } } }
I20260618 22:33:17.492262 23733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ec0714c8d524b5ab5922859329d37d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ec0714c8d524b5ab5922859329d37d6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40287 } } }
I20260618 22:33:17.492527 23734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:17.492631 23733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:17.493538 23740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:17.498713 23740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:17.506644 23740 catalog_manager.cc:1347] Generated new cluster ID: 92c853440e994cad8aa26d23c66b84f7
I20260618 22:33:17.506747 23740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:17.531256 23740 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:17.531767 23740 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:17.543041 23740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6: Generated new TSK 0
I20260618 22:33:17.544031 23740 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:33:17.556855 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40287
--builtin_ntp_servers=127.18.150.148:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260618 22:33:17.870767 23751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:17.871037 23751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:17.871122 23751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:17.878168 23751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:17.878268 23751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:17.878381 23751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:33:17.887316 23751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:17.888527 23751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:17.890272 23751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:17.902733 23759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:17.902761 23756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:17.902797 23757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:17.903107 23751 server_base.cc:1034] running on GCE node
I20260618 22:33:17.903851 23751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:17.904726 23751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:17.905918 23751 hybrid_clock.cc:648] HybridClock initialized: now 1781821997905856 us; error 48 us; skew 500 ppm
I20260618 22:33:17.906270 23751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:33:17.908408 23751 webserver.cc:460] Webserver started at http://127.18.150.129:43683/ using document root <none> and password file <none>
I20260618 22:33:17.909044 23751 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:17.909140 23751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:17.909436 23751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:17.912180 23751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "117bbe4237194e5f8b078cea62948f19"
format_stamp: "Formatted at 2026-06-18 22:33:17 on dist-test-slave-h7jz"
server_key: "ca38dbc333c9e701c2d416db7ea502a6"
server_key_iv: "2b6331debae3aa7667076418564ed1e4"
server_key_version: "encryptionkey@0"
I20260618 22:33:17.912935 23751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "117bbe4237194e5f8b078cea62948f19"
format_stamp: "Formatted at 2026-06-18 22:33:17 on dist-test-slave-h7jz"
server_key: "ca38dbc333c9e701c2d416db7ea502a6"
server_key_iv: "2b6331debae3aa7667076418564ed1e4"
server_key_version: "encryptionkey@0"
I20260618 22:33:17.917834 23751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260618 22:33:17.921288 23765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:17.922571 23751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:17.922715 23751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "117bbe4237194e5f8b078cea62948f19"
format_stamp: "Formatted at 2026-06-18 22:33:17 on dist-test-slave-h7jz"
server_key: "ca38dbc333c9e701c2d416db7ea502a6"
server_key_iv: "2b6331debae3aa7667076418564ed1e4"
server_key_version: "encryptionkey@0"
I20260618 22:33:17.922902 23751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:17.928263 23751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:17.929013 23751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:17.929255 23751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:17.930243 23751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:17.931584 23751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:17.931665 23751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:17.931741 23751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:17.931792 23751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:17.960098 23751 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:38101
I20260618 22:33:17.960167 23877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:38101 every 8 connection(s)
I20260618 22:33:17.961355 23751 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:33:17.963608 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 23751
I20260618 22:33:17.963788 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:33:17.964424 19034 external_mini_cluster.cc:1428] Setting key e012f1e919e3cd2be8fe3cf1548f288c
I20260618 22:33:17.966846 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40287
--builtin_ntp_servers=127.18.150.148:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260618 22:33:17.969661 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:17.963891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:41259 (local address 127.18.150.190:40287)
0618 22:33:17.964101 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:17.964108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:17.965790 (+ 1682us) server_negotiation.cc:407] Connection header received
0618 22:33:17.967500 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:17.967506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:17.967566 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:17.967670 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:17.968977 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:17.968982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:17.968986 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:17.969007 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:17.969044 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:17.969058 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:17.969168 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:17.969352 (+ 184us) server_negotiation.cc:299] Negotiation successful
0618 22:33:17.969407 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260618 22:33:17.970638 23880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:17.964517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:40287 (local address 127.18.150.129:41259)
0618 22:33:17.965626 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:17.965677 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:17.967230 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:17.967874 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:17.967883 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:17.967980 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:17.968663 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:33:17.968666 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:17.968707 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:33:17.968801 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:17.969166 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:17.969179 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:17.969185 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:33:17.969401 (+ 216us) client_negotiation.cc:240] Negotiation successful
0618 22:33:17.969635 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":353,"threads_started":1}
I20260618 22:33:17.972260 23878 heartbeater.cc:347] Connected to a master server at 127.18.150.190:40287
I20260618 22:33:17.972538 23878 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:17.973109 23878 heartbeater.cc:511] Master 127.18.150.190:40287 requested a full tablet report, sending...
I20260618 22:33:17.975109 23692 ts_manager.cc:194] Registered new tserver with Master: 117bbe4237194e5f8b078cea62948f19 (127.18.150.129:38101)
W20260618 22:33:18.274950 23881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:18.275236 23881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:18.275367 23881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:18.282377 23881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:18.282480 23881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:18.282622 23881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:33:18.291584 23881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:18.292810 23881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:18.294684 23881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:18.306766 23887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:18.306720 23889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:18.306810 23886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:18.307323 23881 server_base.cc:1034] running on GCE node
I20260618 22:33:18.307897 23881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:18.308745 23881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:18.309967 23881 hybrid_clock.cc:648] HybridClock initialized: now 1781821998309892 us; error 46 us; skew 500 ppm
I20260618 22:33:18.310269 23881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:33:18.312224 23881 webserver.cc:460] Webserver started at http://127.18.150.130:34291/ using document root <none> and password file <none>
I20260618 22:33:18.312860 23881 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:18.312955 23881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:18.313265 23881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:18.315964 23881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "a8682990c8a54432ab40393f094eb451"
format_stamp: "Formatted at 2026-06-18 22:33:18 on dist-test-slave-h7jz"
server_key: "7474dd01ba86f8e33827d1a1ea1a6532"
server_key_iv: "4e8a67bf7d4502e35e31cecf7651eb5f"
server_key_version: "encryptionkey@0"
I20260618 22:33:18.316696 23881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "a8682990c8a54432ab40393f094eb451"
format_stamp: "Formatted at 2026-06-18 22:33:18 on dist-test-slave-h7jz"
server_key: "7474dd01ba86f8e33827d1a1ea1a6532"
server_key_iv: "4e8a67bf7d4502e35e31cecf7651eb5f"
server_key_version: "encryptionkey@0"
I20260618 22:33:18.321285 23881 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260618 22:33:18.324438 23895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:18.325628 23881 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260618 22:33:18.325762 23881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "a8682990c8a54432ab40393f094eb451"
format_stamp: "Formatted at 2026-06-18 22:33:18 on dist-test-slave-h7jz"
server_key: "7474dd01ba86f8e33827d1a1ea1a6532"
server_key_iv: "4e8a67bf7d4502e35e31cecf7651eb5f"
server_key_version: "encryptionkey@0"
I20260618 22:33:18.325959 23881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:18.331010 23881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:18.331722 23881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:18.331946 23881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:18.332741 23881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:18.334088 23881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:18.334183 23881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:18.334282 23881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:18.334398 23881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:18.362100 23881 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:41371
I20260618 22:33:18.362149 24007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:41371 every 8 connection(s)
I20260618 22:33:18.363329 23881 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:33:18.364742 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 23881
I20260618 22:33:18.364882 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:33:18.365355 19034 external_mini_cluster.cc:1428] Setting key 5e5ef72b90acd2c9120dfb8bc0304f18
I20260618 22:33:18.367913 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40287
--builtin_ntp_servers=127.18.150.148:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260618 22:33:18.371071 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.365675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:58443 (local address 127.18.150.190:40287)
0618 22:33:18.365867 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:18.365873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:18.367474 (+ 1601us) server_negotiation.cc:407] Connection header received
0618 22:33:18.369120 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:18.369126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:18.369182 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:18.369281 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:18.370438 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:18.370443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:18.370448 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:18.370466 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:18.370501 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:18.370515 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:18.370628 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:18.370755 (+ 127us) server_negotiation.cc:299] Negotiation successful
0618 22:33:18.370808 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:33:18.371870 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.366324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:40287 (local address 127.18.150.130:58443)
0618 22:33:18.367312 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:18.367360 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:18.368867 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:18.369444 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:18.369451 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:18.369538 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:18.370173 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:33:18.370175 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:18.370214 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:33:18.370295 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:18.370592 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:18.370597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:18.370601 (+ 4us) client_negotiation.cc:769] Sending connection context
0618 22:33:18.370796 (+ 195us) client_negotiation.cc:240] Negotiation successful
0618 22:33:18.370978 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":294,"threads_started":1}
I20260618 22:33:18.373057 24008 heartbeater.cc:347] Connected to a master server at 127.18.150.190:40287
I20260618 22:33:18.373318 24008 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:18.373844 24008 heartbeater.cc:511] Master 127.18.150.190:40287 requested a full tablet report, sending...
I20260618 22:33:18.375075 23692 ts_manager.cc:194] Registered new tserver with Master: a8682990c8a54432ab40393f094eb451 (127.18.150.130:41371)
W20260618 22:33:18.680552 24011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:18.680876 24011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:18.681010 24011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:18.688197 24011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:18.688298 24011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:18.688454 24011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:33:18.697750 24011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:18.699049 24011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:18.700752 24011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:18.713032 24016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:18.712965 24019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:18.713001 24017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:18.713994 24011 server_base.cc:1034] running on GCE node
I20260618 22:33:18.714596 24011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:18.715505 24011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:18.716739 24011 hybrid_clock.cc:648] HybridClock initialized: now 1781821998716680 us; error 38 us; skew 500 ppm
I20260618 22:33:18.717069 24011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:33:18.719216 24011 webserver.cc:460] Webserver started at http://127.18.150.131:41137/ using document root <none> and password file <none>
I20260618 22:33:18.719892 24011 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:18.719981 24011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:18.720284 24011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:18.723095 24011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "258da22a91b14b969c78618a77863e60"
format_stamp: "Formatted at 2026-06-18 22:33:18 on dist-test-slave-h7jz"
server_key: "bbafb374888658453451b8aacc1d0b9c"
server_key_iv: "8a924de21066d48fb0395741643d1949"
server_key_version: "encryptionkey@0"
I20260618 22:33:18.723852 24011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "258da22a91b14b969c78618a77863e60"
format_stamp: "Formatted at 2026-06-18 22:33:18 on dist-test-slave-h7jz"
server_key: "bbafb374888658453451b8aacc1d0b9c"
server_key_iv: "8a924de21066d48fb0395741643d1949"
server_key_version: "encryptionkey@0"
I20260618 22:33:18.728895 24011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260618 22:33:18.732285 24025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:18.733618 24011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260618 22:33:18.733773 24011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "258da22a91b14b969c78618a77863e60"
format_stamp: "Formatted at 2026-06-18 22:33:18 on dist-test-slave-h7jz"
server_key: "bbafb374888658453451b8aacc1d0b9c"
server_key_iv: "8a924de21066d48fb0395741643d1949"
server_key_version: "encryptionkey@0"
I20260618 22:33:18.734001 24011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:18.739538 24011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:18.740285 24011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:18.740505 24011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:18.741320 24011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:18.742717 24011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:18.742798 24011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:18.742879 24011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:18.742941 24011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:18.772253 24011 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:34141
I20260618 22:33:18.772297 24137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:34141 every 8 connection(s)
I20260618 22:33:18.773525 24011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:33:18.781206 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.775986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:51101 (local address 127.18.150.190:40287)
0618 22:33:18.776153 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:18.776158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:18.777493 (+ 1335us) server_negotiation.cc:407] Connection header received
0618 22:33:18.779188 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:18.779192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:18.779243 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:18.779333 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:18.780562 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:18.780568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:18.780572 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:18.780591 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:18.780625 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:18.780637 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:18.780746 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:18.780930 (+ 184us) server_negotiation.cc:299] Negotiation successful
0618 22:33:18.780986 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:33:18.782096 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.776514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:40287 (local address 127.18.150.131:51101)
0618 22:33:18.777368 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:18.777411 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:18.778908 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:18.779533 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:18.779541 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:18.779642 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:18.780261 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:33:18.780263 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:18.780299 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:33:18.780386 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:18.780731 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:18.780737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:18.780742 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:33:18.780941 (+ 199us) client_negotiation.cc:240] Negotiation successful
0618 22:33:18.781142 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":245,"threads_started":1}
I20260618 22:33:18.783440 24138 heartbeater.cc:347] Connected to a master server at 127.18.150.190:40287
I20260618 22:33:18.783717 24138 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:18.783926 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 24011
I20260618 22:33:18.784086 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:33:18.784317 24138 heartbeater.cc:511] Master 127.18.150.190:40287 requested a full tablet report, sending...
I20260618 22:33:18.784628 19034 external_mini_cluster.cc:1428] Setting key 9185995ea2ac726f1e7b9280e63721b6
I20260618 22:33:18.786183 23691 ts_manager.cc:194] Registered new tserver with Master: 258da22a91b14b969c78618a77863e60 (127.18.150.131:34141)
I20260618 22:33:18.788674 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:33:18.795219 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.792974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46998 (local address 127.18.150.190:40287)
0618 22:33:18.793155 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:18.793160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:18.793609 (+ 449us) server_negotiation.cc:407] Connection header received
0618 22:33:18.793747 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:18.793751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:18.793801 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:18.793885 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:18.794708 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:18.794713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:18.794716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:18.794730 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:18.794760 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:18.794772 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:18.794875 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:18.794935 (+ 60us) server_negotiation.cc:299] Negotiation successful
0618 22:33:18.794981 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:33:18.799034 23692 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:33:18.805480 23692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:33:18.821880 24150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.816744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:34141 (local address 127.0.0.1:38220)
0618 22:33:18.817568 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:18.817606 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:18.817751 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:18.819082 (+ 1331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:18.819089 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:18.819187 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:18.820000 (+ 813us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:33:18.820004 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:18.820035 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:33:18.820119 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:18.821198 (+ 1079us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:18.821212 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:18.821220 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:33:18.821378 (+ 158us) client_negotiation.cc:240] Negotiation successful
0618 22:33:18.821463 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":446,"threads_started":1}
I20260618 22:33:18.822670 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.817513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55118 (local address 127.18.150.129:38101)
0618 22:33:18.818223 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:18.818232 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:18.818537 (+ 305us) server_negotiation.cc:407] Connection header received
0618 22:33:18.818608 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:18.818616 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:18.818859 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:18.819008 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:18.820712 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:18.820721 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:18.820735 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:18.820753 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:18.820811 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:18.820830 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:18.820970 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:18.822291 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0618 22:33:18.822398 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":432,"threads_started":1}
I20260618 22:33:18.822840 24152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.816959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38220 (local address 127.18.150.131:34141)
0618 22:33:18.818641 (+ 1682us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:18.818648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:18.818673 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:33:18.818737 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:18.818744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:18.818887 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:18.819639 (+ 752us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:18.820976 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:18.820984 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:18.820996 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:18.821011 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:18.821058 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:18.821076 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:18.822376 (+ 1300us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:18.822431 (+ 55us) server_negotiation.cc:299] Negotiation successful
0618 22:33:18.822504 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1528,"thread_start_us":1902,"threads_started":1}
I20260618 22:33:18.823004 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.818994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46860 (local address 127.18.150.130:41371)
0618 22:33:18.820584 (+ 1590us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:18.820589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:18.820614 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:33:18.820674 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:18.820681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:18.820920 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:18.821095 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:18.822316 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:18.822321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:18.822330 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:18.822347 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:18.822377 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:18.822389 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:18.822524 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:18.822587 (+ 63us) server_negotiation.cc:299] Negotiation successful
0618 22:33:18.822670 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1472,"thread_start_us":1143,"threads_started":1}
I20260618 22:33:18.823009 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.818454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:41371 (local address 127.0.0.1:46860)
0618 22:33:18.819426 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:18.819444 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:18.819540 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:18.821096 (+ 1556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:18.821101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:18.821122 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:18.822077 (+ 955us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:33:18.822081 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:18.822096 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:33:18.822139 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:18.822474 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:18.822479 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:18.822483 (+ 4us) client_negotiation.cc:769] Sending connection context
0618 22:33:18.822603 (+ 120us) client_negotiation.cc:240] Negotiation successful
0618 22:33:18.822649 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"thread_start_us":749,"threads_started":1}
I20260618 22:33:18.824086 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.816950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:38101 (local address 127.0.0.1:55118)
0618 22:33:18.818319 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:18.818336 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:18.818432 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:18.819893 (+ 1461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:18.819898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:18.819924 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:18.820501 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:33:18.820504 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:18.820522 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:33:18.820565 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:18.821687 (+ 1122us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:18.821692 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:18.821696 (+ 4us) client_negotiation.cc:769] Sending connection context
0618 22:33:18.821839 (+ 143us) client_negotiation.cc:240] Negotiation successful
0618 22:33:18.821889 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1251,"thread_start_us":1162,"threads_started":1}
I20260618 22:33:18.826467 24073 tablet_service.cc:1467] Processing CreateTablet for tablet 642aec10ab7a4def82643ae21ffc31d6 (DEFAULT_TABLE table=test-table [id=7e0d79d6ba1344c89202edf8d9c01bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:18.826499 23943 tablet_service.cc:1467] Processing CreateTablet for tablet 642aec10ab7a4def82643ae21ffc31d6 (DEFAULT_TABLE table=test-table [id=7e0d79d6ba1344c89202edf8d9c01bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:18.827687 24073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642aec10ab7a4def82643ae21ffc31d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:18.827304 23813 tablet_service.cc:1467] Processing CreateTablet for tablet 642aec10ab7a4def82643ae21ffc31d6 (DEFAULT_TABLE table=test-table [id=7e0d79d6ba1344c89202edf8d9c01bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:18.827759 23943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642aec10ab7a4def82643ae21ffc31d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:18.828402 23813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642aec10ab7a4def82643ae21ffc31d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:18.838587 24156 tablet_bootstrap.cc:492] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60: Bootstrap starting.
I20260618 22:33:18.841547 24157 tablet_bootstrap.cc:492] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: Bootstrap starting.
I20260618 22:33:18.842206 24156 tablet_bootstrap.cc:654] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:18.842974 24158 tablet_bootstrap.cc:492] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19: Bootstrap starting.
I20260618 22:33:18.843545 24156 log.cc:824] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:18.844759 24157 tablet_bootstrap.cc:654] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:18.845736 24158 tablet_bootstrap.cc:654] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:18.845885 24157 log.cc:824] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:18.846673 24156 tablet_bootstrap.cc:492] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60: No bootstrap required, opened a new log
I20260618 22:33:18.846791 24158 log.cc:824] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:18.846930 24156 ts_tablet_manager.cc:1397] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260618 22:33:18.848896 24157 tablet_bootstrap.cc:492] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: No bootstrap required, opened a new log
I20260618 22:33:18.849151 24157 ts_tablet_manager.cc:1397] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260618 22:33:18.854000 24158 tablet_bootstrap.cc:492] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19: No bootstrap required, opened a new log
I20260618 22:33:18.854249 24158 ts_tablet_manager.cc:1397] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.006s
I20260618 22:33:18.854043 24156 raft_consensus.cc:348] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "258da22a91b14b969c78618a77863e60" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34141 } } peers { permanent_uuid: "117bbe4237194e5f8b078cea62948f19" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38101 } } peers { permanent_uuid: "a8682990c8a54432ab40393f094eb451" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41371 } }
I20260618 22:33:18.854389 24156 raft_consensus.cc:374] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:18.854481 24156 raft_consensus.cc:729] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 258da22a91b14b969c78618a77863e60, State: Initialized, Role: FOLLOWER
I20260618 22:33:18.855012 24156 consensus_queue.cc:260] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "258da22a91b14b969c78618a77863e60" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34141 } } peers { permanent_uuid: "117bbe4237194e5f8b078cea62948f19" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38101 } } peers { permanent_uuid: "a8682990c8a54432ab40393f094eb451" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41371 } }
I20260618 22:33:18.856580 24156 ts_tablet_manager.cc:1428] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60: Time spent starting tablet: real 0.010s user 0.007s sys 0.002s
I20260618 22:33:18.856973 24138 heartbeater.cc:503] Master 127.18.150.190:40287 was elected leader, sending a full tablet report...
I20260618 22:33:18.858671 24157 raft_consensus.cc:348] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "258da22a91b14b969c78618a77863e60" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34141 } } peers { permanent_uuid: "117bbe4237194e5f8b078cea62948f19" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38101 } } peers { permanent_uuid: "a8682990c8a54432ab40393f094eb451" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41371 } }
I20260618 22:33:18.859076 24157 raft_consensus.cc:374] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:18.859160 24157 raft_consensus.cc:729] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8682990c8a54432ab40393f094eb451, State: Initialized, Role: FOLLOWER
I20260618 22:33:18.859735 24157 consensus_queue.cc:260] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "258da22a91b14b969c78618a77863e60" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34141 } } peers { permanent_uuid: "117bbe4237194e5f8b078cea62948f19" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38101 } } peers { permanent_uuid: "a8682990c8a54432ab40393f094eb451" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41371 } }
I20260618 22:33:18.860746 24008 heartbeater.cc:503] Master 127.18.150.190:40287 was elected leader, sending a full tablet report...
I20260618 22:33:18.861269 24157 ts_tablet_manager.cc:1428] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260618 22:33:18.863178 24158 raft_consensus.cc:348] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "258da22a91b14b969c78618a77863e60" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34141 } } peers { permanent_uuid: "117bbe4237194e5f8b078cea62948f19" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38101 } } peers { permanent_uuid: "a8682990c8a54432ab40393f094eb451" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41371 } }
I20260618 22:33:18.863538 24158 raft_consensus.cc:374] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:18.863627 24158 raft_consensus.cc:729] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117bbe4237194e5f8b078cea62948f19, State: Initialized, Role: FOLLOWER
I20260618 22:33:18.864187 24158 consensus_queue.cc:260] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "258da22a91b14b969c78618a77863e60" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34141 } } peers { permanent_uuid: "117bbe4237194e5f8b078cea62948f19" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38101 } } peers { permanent_uuid: "a8682990c8a54432ab40393f094eb451" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41371 } }
W20260618 22:33:18.864969 24009 tablet.cc:2369] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:33:18.865201 23878 heartbeater.cc:503] Master 127.18.150.190:40287 was elected leader, sending a full tablet report...
I20260618 22:33:18.865667 24158 ts_tablet_manager.cc:1428] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20260618 22:33:18.928004 24163 raft_consensus.cc:482] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:33:18.928272 24163 raft_consensus.cc:504] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "258da22a91b14b969c78618a77863e60" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34141 } } peers { permanent_uuid: "117bbe4237194e5f8b078cea62948f19" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38101 } } peers { permanent_uuid: "a8682990c8a54432ab40393f094eb451" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41371 } }
I20260618 22:33:18.929874 24163 leader_election.cc:288] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 258da22a91b14b969c78618a77863e60 (127.18.150.131:34141), 117bbe4237194e5f8b078cea62948f19 (127.18.150.129:38101)
I20260618 22:33:18.932739 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.930187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:56051 (local address 127.18.150.129:38101)
0618 22:33:18.930357 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:18.930364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:18.930994 (+ 630us) server_negotiation.cc:407] Connection header received
0618 22:33:18.931142 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:18.931146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:18.931194 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:18.931272 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:18.932192 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:18.932197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:18.932201 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:18.932216 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:18.932244 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:18.932255 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:18.932346 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:18.932437 (+ 91us) server_negotiation.cc:299] Negotiation successful
0618 22:33:18.932486 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:33:18.932828 24152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.930290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:44517 (local address 127.18.150.131:34141)
0618 22:33:18.930438 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:18.930443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:18.931068 (+ 625us) server_negotiation.cc:407] Connection header received
0618 22:33:18.931176 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:18.931181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:18.931232 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:18.931309 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:18.932297 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:18.932301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:18.932304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:18.932317 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:18.932346 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:18.932355 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:18.932452 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:18.932527 (+ 75us) server_negotiation.cc:299] Negotiation successful
0618 22:33:18.932583 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:33:18.932967 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.930125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:38101 (local address 127.18.150.130:56051)
0618 22:33:18.930888 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:18.930907 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:18.931017 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:18.931304 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:18.931309 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:18.931330 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:18.931977 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:33:18.931980 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:18.931997 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:33:18.932042 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:18.932332 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:18.932337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:18.932342 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:33:18.932428 (+ 86us) client_negotiation.cc:240] Negotiation successful
0618 22:33:18.932478 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":507,"threads_started":1}
I20260618 22:33:18.933324 24166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.930175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:34141 (local address 127.18.150.130:44517)
0618 22:33:18.930965 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:18.930983 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:18.931077 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:18.931344 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:18.931348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:18.931368 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:18.932114 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:33:18.932117 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:18.932128 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:33:18.932159 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:18.932430 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:18.932435 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:18.932439 (+ 4us) client_negotiation.cc:769] Sending connection context
0618 22:33:18.932564 (+ 125us) client_negotiation.cc:240] Negotiation successful
0618 22:33:18.932610 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":496,"threads_started":1}
I20260618 22:33:18.933815 23833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642aec10ab7a4def82643ae21ffc31d6" candidate_uuid: "a8682990c8a54432ab40393f094eb451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117bbe4237194e5f8b078cea62948f19" is_pre_election: true
I20260618 22:33:18.934377 23833 raft_consensus.cc:2454] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8682990c8a54432ab40393f094eb451 in term 0.
I20260618 22:33:18.934264 24093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642aec10ab7a4def82643ae21ffc31d6" candidate_uuid: "a8682990c8a54432ab40393f094eb451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "258da22a91b14b969c78618a77863e60" is_pre_election: true
I20260618 22:33:18.934760 24093 raft_consensus.cc:2454] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8682990c8a54432ab40393f094eb451 in term 0.
I20260618 22:33:18.935010 23897 leader_election.cc:302] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117bbe4237194e5f8b078cea62948f19, a8682990c8a54432ab40393f094eb451; no voters:
I20260618 22:33:18.935380 24163 raft_consensus.cc:2781] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:33:18.935515 24163 raft_consensus.cc:482] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:33:18.935608 24163 raft_consensus.cc:3037] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:18.937650 24163 raft_consensus.cc:504] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "258da22a91b14b969c78618a77863e60" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34141 } } peers { permanent_uuid: "117bbe4237194e5f8b078cea62948f19" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38101 } } peers { permanent_uuid: "a8682990c8a54432ab40393f094eb451" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41371 } }
I20260618 22:33:18.938205 24163 leader_election.cc:288] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [CANDIDATE]: Term 1 election: Requested vote from peers 258da22a91b14b969c78618a77863e60 (127.18.150.131:34141), 117bbe4237194e5f8b078cea62948f19 (127.18.150.129:38101)
I20260618 22:33:18.938594 24093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642aec10ab7a4def82643ae21ffc31d6" candidate_uuid: "a8682990c8a54432ab40393f094eb451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "258da22a91b14b969c78618a77863e60"
I20260618 22:33:18.938594 23833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642aec10ab7a4def82643ae21ffc31d6" candidate_uuid: "a8682990c8a54432ab40393f094eb451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117bbe4237194e5f8b078cea62948f19"
I20260618 22:33:18.938781 23833 raft_consensus.cc:3037] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:18.938783 24093 raft_consensus.cc:3037] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:18.941764 24093 raft_consensus.cc:2454] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8682990c8a54432ab40393f094eb451 in term 1.
I20260618 22:33:18.941764 23833 raft_consensus.cc:2454] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8682990c8a54432ab40393f094eb451 in term 1.
I20260618 22:33:18.942131 23897 leader_election.cc:302] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117bbe4237194e5f8b078cea62948f19, a8682990c8a54432ab40393f094eb451; no voters:
I20260618 22:33:18.942394 24163 raft_consensus.cc:2781] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:18.942797 24163 raft_consensus.cc:686] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 1 LEADER]: Becoming Leader. State: Replica: a8682990c8a54432ab40393f094eb451, State: Running, Role: LEADER
I20260618 22:33:18.943234 24163 consensus_queue.cc:237] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "258da22a91b14b969c78618a77863e60" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34141 } } peers { permanent_uuid: "117bbe4237194e5f8b078cea62948f19" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38101 } } peers { permanent_uuid: "a8682990c8a54432ab40393f094eb451" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41371 } }
I20260618 22:33:18.947350 23692 catalog_manager.cc:5495] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8682990c8a54432ab40393f094eb451 (127.18.150.130). New cstate: current_term: 1 leader_uuid: "a8682990c8a54432ab40393f094eb451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "258da22a91b14b969c78618a77863e60" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117bbe4237194e5f8b078cea62948f19" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8682990c8a54432ab40393f094eb451" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41371 } health_report { overall_health: HEALTHY } } }
W20260618 22:33:18.963353 23879 tablet.cc:2369] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:33:18.976749 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:18.975209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.18.150.130:41371)
0618 22:33:18.975358 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:18.975363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:18.975378 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:33:18.975457 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:18.975460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:18.975543 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:18.975619 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:18.976324 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:18.976328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:18.976332 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:18.976345 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:18.976374 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:18.976385 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:18.976478 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:18.976536 (+ 58us) server_negotiation.cc:299] Negotiation successful
0618 22:33:18.976579 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:33:18.984174 23833 raft_consensus.cc:1261] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19 [term 1 FOLLOWER]: Refusing update from remote peer a8682990c8a54432ab40393f094eb451: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:18.984175 24093 raft_consensus.cc:1261] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60 [term 1 FOLLOWER]: Refusing update from remote peer a8682990c8a54432ab40393f094eb451: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:18.985149 24167 consensus_queue.cc:1035] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117bbe4237194e5f8b078cea62948f19" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:18.985498 24163 consensus_queue.cc:1035] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "258da22a91b14b969c78618a77863e60" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:19.010308 23691 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:19.010529 23691 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:19.013662 23691 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ec0714c8d524b5ab5922859329d37d6: Sending DeleteTablet for 3 replicas of tablet 642aec10ab7a4def82643ae21ffc31d6
I20260618 22:33:19.014755 24073 tablet_service.cc:1514] Processing DeleteTablet for tablet 642aec10ab7a4def82643ae21ffc31d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:19 UTC) from {username='slave'} at 127.0.0.1:38220
I20260618 22:33:19.014871 23943 tablet_service.cc:1514] Processing DeleteTablet for tablet 642aec10ab7a4def82643ae21ffc31d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:19 UTC) from {username='slave'} at 127.0.0.1:46860
I20260618 22:33:19.014909 23813 tablet_service.cc:1514] Processing DeleteTablet for tablet 642aec10ab7a4def82643ae21ffc31d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:19 UTC) from {username='slave'} at 127.0.0.1:55118
I20260618 22:33:19.016619 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 23751
I20260618 22:33:19.025672 24180 tablet_replica.cc:330] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60: stopping tablet replica
I20260618 22:33:19.027057 24180 raft_consensus.cc:2229] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:19.027582 24180 raft_consensus.cc:2258] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:19.027690 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:19.018134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55120 (local address 127.18.150.129:38101)
0618 22:33:19.018298 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:19.018305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:19.023695 (+ 5390us) server_negotiation.cc:407] Connection header received
0618 22:33:19.023871 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:19.023876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:19.023982 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:19.024081 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:19.026875 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:19.026881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:19.026886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:19.026906 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:19.026944 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:19.026960 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:19.027050 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:19.027297 (+ 247us) server_negotiation.cc:299] Negotiation successful
0618 22:33:19.027379 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:33:19.028685 23873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:19.031953 24180 ts_tablet_manager.cc:1905] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:19.034600 24181 tablet_replica.cc:330] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: stopping tablet replica
I20260618 22:33:19.034746 24182 tablet_replica.cc:330] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19: stopping tablet replica
I20260618 22:33:19.035650 24181 raft_consensus.cc:2229] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:33:19.036288 24181 raft_consensus.cc:2258] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:19.038506 24181 ts_tablet_manager.cc:1905] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:19.039433 24180 ts_tablet_manager.cc:1918] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:19.039587 24180 log.cc:1196] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/642aec10ab7a4def82643ae21ffc31d6
I20260618 22:33:19.040050 24180 ts_tablet_manager.cc:1939] T 642aec10ab7a4def82643ae21ffc31d6 P 258da22a91b14b969c78618a77863e60: Deleting consensus metadata
I20260618 22:33:19.041781 23678 catalog_manager.cc:4841] TS 258da22a91b14b969c78618a77863e60 (127.18.150.131:34141): tablet 642aec10ab7a4def82643ae21ffc31d6 (table test-table [id=7e0d79d6ba1344c89202edf8d9c01bfc]) successfully deleted
I20260618 22:33:19.044497 24181 ts_tablet_manager.cc:1918] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:19.044639 24181 log.cc:1196] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/642aec10ab7a4def82643ae21ffc31d6
I20260618 22:33:19.045058 24181 ts_tablet_manager.cc:1939] T 642aec10ab7a4def82643ae21ffc31d6 P a8682990c8a54432ab40393f094eb451: Deleting consensus metadata
I20260618 22:33:19.046355 23677 catalog_manager.cc:4841] TS a8682990c8a54432ab40393f094eb451 (127.18.150.130:41371): tablet 642aec10ab7a4def82643ae21ffc31d6 (table test-table [id=7e0d79d6ba1344c89202edf8d9c01bfc]) successfully deleted
I20260618 22:33:19.035204 24182 raft_consensus.cc:2229] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:19.150869 24182 raft_consensus.cc:2258] T 642aec10ab7a4def82643ae21ffc31d6 P 117bbe4237194e5f8b078cea62948f19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:19.150979 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 23751
I20260618 22:33:19.164497 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 23881
W20260618 22:33:19.164492 23677 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.150.129:38101 (error 108)
I20260618 22:33:19.166409 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:19.165979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:38101 (local address 127.0.0.1:55134)
0618 22:33:19.166115 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:19.166248 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260618 22:33:19.166633 23677 catalog_manager.cc:4584] TS 117bbe4237194e5f8b078cea62948f19 (127.18.150.129:38101): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 642aec10ab7a4def82643ae21ffc31d6: Network error: Client connection negotiation failed: client connection to 127.18.150.129:38101: connect: Connection refused (error 111)
I20260618 22:33:19.167799 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:19.165548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46870 (local address 127.18.150.130:41371)
0618 22:33:19.165733 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:19.165739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:19.165782 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:33:19.165865 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:19.165871 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:19.165930 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:19.166053 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:19.167192 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:19.167198 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:19.167203 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:19.167226 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:19.167265 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:19.167281 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:19.167407 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:19.167466 (+ 59us) server_negotiation.cc:299] Negotiation successful
0618 22:33:19.167520 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:33:19.168568 24003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:19.220135 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:19.219608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:38101 (local address 127.0.0.1:55150)
0618 22:33:19.219832 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:19.219937 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260618 22:33:19.221108 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:19.220766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:38101 (local address 127.0.0.1:55160)
0618 22:33:19.220898 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:19.220983 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260618 22:33:19.274067 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 23881
I20260618 22:33:19.281570 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:19.281134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:38101 (local address 127.0.0.1:55166)
0618 22:33:19.281320 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:19.281419 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260618 22:33:19.282609 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:19.282282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:38101 (local address 127.0.0.1:55170)
0618 22:33:19.282402 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:19.282476 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260618 22:33:19.289547 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 24011
I20260618 22:33:19.291939 24152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:19.290146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38234 (local address 127.18.150.131:34141)
0618 22:33:19.290298 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:19.290302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:19.290354 (+ 52us) server_negotiation.cc:407] Connection header received
0618 22:33:19.290532 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:19.290536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:19.290626 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:19.290705 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:19.291423 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:19.291429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:19.291434 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:19.291452 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:19.291488 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:19.291503 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:19.291602 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:19.291694 (+ 92us) server_negotiation.cc:299] Negotiation successful
0618 22:33:19.291741 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:33:19.292484 24133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:19.362840 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:19.362319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:38101 (local address 127.0.0.1:55186)
0618 22:33:19.362549 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:19.362662 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260618 22:33:19.363921 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:19.363481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:38101 (local address 127.0.0.1:55194)
0618 22:33:19.363692 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:19.363789 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:38101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260618 22:33:19.377269 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 24011
I20260618 22:33:19.389457 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 23661
I20260618 22:33:19.390535 23722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:19.491662 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 23661
2026-06-18T22:33:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2468 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260618 22:33:19.527027 19034 mini_oidc.cc:152] Starting JWKS server
I20260618 22:33:19.529580 19034 webserver.cc:460] Webserver started at https://127.0.0.1:43177/ using document root <none> and password file <none>
I20260618 22:33:19.529886 19034 mini_oidc.cc:189] Starting OIDC Discovery server
I20260618 22:33:19.530529 19034 webserver.cc:460] Webserver started at http://127.0.0.1:45867/ using document root <none> and password file <none>
2026-06-18T22:33:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:19Z Disabled control of system clock
I20260618 22:33:19.552908 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38335
--webserver_interface=127.18.150.190
--webserver_port=0
--builtin_ntp_servers=127.18.150.148:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:38335
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43177/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/testchainca.cert with env {}
W20260618 22:33:19.857560 24194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:19.857862 24194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:19.858029 24194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:19.865276 24194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:19.865401 24194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:19.865463 24194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:19.865536 24194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:19.865597 24194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:33:19.865676 24194 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:33:19.865728 24194 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43177/jwks.json
I20260618 22:33:19.875535 24194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:38335
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43177/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:19.876807 24194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:19.878034 24194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:19.888988 24200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:19.889060 24202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:19.889031 24199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:19.889662 24194 server_base.cc:1034] running on GCE node
I20260618 22:33:19.890561 24194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:19.891903 24194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:19.893121 24194 hybrid_clock.cc:648] HybridClock initialized: now 1781821999893063 us; error 48 us; skew 500 ppm
I20260618 22:33:19.893443 24194 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:33:19.899307 24194 webserver.cc:460] Webserver started at http://127.18.150.190:39515/ using document root <none> and password file <none>
I20260618 22:33:19.899996 24194 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:19.900099 24194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:19.900401 24194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:19.903357 24194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "d35ff84f29e94006a3e8fc2146d0fef3"
format_stamp: "Formatted at 2026-06-18 22:33:19 on dist-test-slave-h7jz"
server_key: "ec6795c4015e2fd04d7e573c84452831"
server_key_iv: "5e6f3ab195a055d9baaf523f5ac6b92e"
server_key_version: "encryptionkey@0"
I20260618 22:33:19.904137 24194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "d35ff84f29e94006a3e8fc2146d0fef3"
format_stamp: "Formatted at 2026-06-18 22:33:19 on dist-test-slave-h7jz"
server_key: "ec6795c4015e2fd04d7e573c84452831"
server_key_iv: "5e6f3ab195a055d9baaf523f5ac6b92e"
server_key_version: "encryptionkey@0"
I20260618 22:33:19.909092 24194 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260618 22:33:19.912431 24208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:19.913806 24194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:33:19.914038 24194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "d35ff84f29e94006a3e8fc2146d0fef3"
format_stamp: "Formatted at 2026-06-18 22:33:19 on dist-test-slave-h7jz"
server_key: "ec6795c4015e2fd04d7e573c84452831"
server_key_iv: "5e6f3ab195a055d9baaf523f5ac6b92e"
server_key_version: "encryptionkey@0"
I20260618 22:33:19.914237 24194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:19.935158 24194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:19.936048 24194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:19.936343 24194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:19.953001 24194 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:38335
I20260618 22:33:19.953022 24259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:38335 every 8 connection(s)
I20260618 22:33:19.954381 24194 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:19.957887 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 24194
I20260618 22:33:19.957962 24260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:19.958112 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:19.958704 19034 external_mini_cluster.cc:1428] Setting key c64dbfee2b7405fa67547d16ae6f021b
I20260618 22:33:19.970769 24260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3: Bootstrap starting.
I20260618 22:33:19.972689 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:19.960478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.18.150.190:38335)
0618 22:33:19.961481 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:19.961492 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:19.961534 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:33:19.963537 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:19.963570 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:19.964086 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:19.964452 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:19.965056 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:19.966343 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:19.968240 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:19.968973 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:19.969106 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:19.970325 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:19.970332 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:19.970347 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:19.970890 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:19.970986 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:19.971011 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:19.971112 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:19.971341 (+ 229us) server_negotiation.cc:299] Negotiation successful
0618 22:33:19.971708 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":352,"threads_started":1}
I20260618 22:33:19.974257 24260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:19.975351 24260 log.cc:824] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:19.976555 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:33:19.978024 24260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3: No bootstrap required, opened a new log
I20260618 22:33:19.984782 24260 raft_consensus.cc:348] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35ff84f29e94006a3e8fc2146d0fef3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38335 } }
I20260618 22:33:19.985131 24260 raft_consensus.cc:374] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:19.985234 24260 raft_consensus.cc:729] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35ff84f29e94006a3e8fc2146d0fef3, State: Initialized, Role: FOLLOWER
I20260618 22:33:19.985810 24260 consensus_queue.cc:260] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35ff84f29e94006a3e8fc2146d0fef3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38335 } }
I20260618 22:33:19.986045 24260 raft_consensus.cc:388] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:19.986172 24260 raft_consensus.cc:482] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:19.986331 24260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:19.988372 24260 raft_consensus.cc:504] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35ff84f29e94006a3e8fc2146d0fef3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38335 } }
I20260618 22:33:19.988912 24260 leader_election.cc:302] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d35ff84f29e94006a3e8fc2146d0fef3; no voters:
I20260618 22:33:19.989464 24260 leader_election.cc:288] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:19.989584 24265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:19.990281 24265 raft_consensus.cc:686] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [term 1 LEADER]: Becoming Leader. State: Replica: d35ff84f29e94006a3e8fc2146d0fef3, State: Running, Role: LEADER
I20260618 22:33:19.990782 24265 consensus_queue.cc:237] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35ff84f29e94006a3e8fc2146d0fef3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38335 } }
I20260618 22:33:19.991397 24260 sys_catalog.cc:564] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:19.993299 24266 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d35ff84f29e94006a3e8fc2146d0fef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35ff84f29e94006a3e8fc2146d0fef3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38335 } } }
I20260618 22:33:19.993671 24266 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:19.994490 24267 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d35ff84f29e94006a3e8fc2146d0fef3. Latest consensus state: current_term: 1 leader_uuid: "d35ff84f29e94006a3e8fc2146d0fef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35ff84f29e94006a3e8fc2146d0fef3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38335 } } }
I20260618 22:33:19.994784 24267 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:19.995697 24272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:20.000267 24272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:20.008745 24272 catalog_manager.cc:1347] Generated new cluster ID: 8df66a2d13bf41a794233f58ec27333a
I20260618 22:33:20.008852 24272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:20.051692 24272 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:20.052983 24272 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:20.059733 24272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d35ff84f29e94006a3e8fc2146d0fef3: Generated new TSK 0
I20260618 22:33:20.060742 24272 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260618 22:33:20.106207 24290 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260618 22:33:20.106355 24290 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260618 22:33:20.107586 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:20.086105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.18.150.190:38335)
0618 22:33:20.086292 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:20.086299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:20.086742 (+ 443us) server_negotiation.cc:407] Connection header received
0618 22:33:20.087028 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:20.087034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:20.087138 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:20.087286 (+ 148us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:33:20.087800 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:20.088908 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:20.090309 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:20.091009 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:20.091130 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:20.091675 (+ 545us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:33:20.106913 (+ 15238us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0618 22:33:20.107285 (+ 372us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35608: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260618 22:33:20.107715 24263 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35608: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260618 22:33:20.109213 24289 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:20.085977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38335 (local address 127.0.0.1:35608)
0618 22:33:20.086646 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:20.086662 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:20.086813 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:20.087330 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:20.087333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:20.087342 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:33:20.087666 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:20.087676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:20.089065 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:20.089069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:20.090173 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:20.090186 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:20.091180 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:20.091185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:20.091326 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:20.091385 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:33:20.107218 (+ 15833us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0618 22:33:20.107491 (+ 273us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.150.190:38335: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":357,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.150.190:38335: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260618 22:33:20.110795 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 24194
I20260618 22:33:20.111922 24255 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:20.205319 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 24194
2026-06-18T22:33:20Z chronyd exiting
I20260618 22:33:20.251446 19034 test_util.cc:175] -----------------------------------------------
I20260618 22:33:20.251593 19034 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821960891835-19034-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (728 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260618 22:33:20.255237 19034 mini_oidc.cc:152] Starting JWKS server
I20260618 22:33:20.257004 19034 webserver.cc:460] Webserver started at https://127.0.0.1:40353/ using document root <none> and password file <none>
I20260618 22:33:20.257182 19034 mini_oidc.cc:189] Starting OIDC Discovery server
I20260618 22:33:20.257737 19034 webserver.cc:460] Webserver started at http://127.0.0.1:33225/ using document root <none> and password file <none>
2026-06-18T22:33:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:20Z Disabled control of system clock
I20260618 22:33:20.280898 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:40107
--webserver_interface=127.18.150.190
--webserver_port=0
--builtin_ntp_servers=127.18.150.148:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:40107
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40353/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/testchainca.cert with env {}
W20260618 22:33:20.591606 24297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:20.591912 24297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:20.592056 24297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:20.599296 24297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:20.599423 24297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:20.599501 24297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:20.599587 24297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:20.599651 24297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:33:20.599741 24297 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:33:20.599797 24297 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40353/jwks.json
I20260618 22:33:20.609689 24297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:40107
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:40107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40353/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:20.610987 24297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:20.612274 24297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:20.623561 24302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:20.623603 24305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:20.623615 24303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:20.624486 24297 server_base.cc:1034] running on GCE node
I20260618 22:33:20.625218 24297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:20.626601 24297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:20.627817 24297 hybrid_clock.cc:648] HybridClock initialized: now 1781822000627756 us; error 44 us; skew 500 ppm
I20260618 22:33:20.628161 24297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:33:20.630321 24297 webserver.cc:460] Webserver started at http://127.18.150.190:35489/ using document root <none> and password file <none>
I20260618 22:33:20.630964 24297 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:20.631060 24297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:20.631348 24297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:20.634116 24297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "b46b584d5f61438495ac109bca99039c"
format_stamp: "Formatted at 2026-06-18 22:33:20 on dist-test-slave-h7jz"
server_key: "e92ea5eed5a092d15b2a5af0ab4717b9"
server_key_iv: "11b421b6312c84bd6c50e6f2d0558c0d"
server_key_version: "encryptionkey@0"
I20260618 22:33:20.634855 24297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "b46b584d5f61438495ac109bca99039c"
format_stamp: "Formatted at 2026-06-18 22:33:20 on dist-test-slave-h7jz"
server_key: "e92ea5eed5a092d15b2a5af0ab4717b9"
server_key_iv: "11b421b6312c84bd6c50e6f2d0558c0d"
server_key_version: "encryptionkey@0"
I20260618 22:33:20.639736 24297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:33:20.643119 24311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:20.644474 24297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:33:20.644654 24297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "b46b584d5f61438495ac109bca99039c"
format_stamp: "Formatted at 2026-06-18 22:33:20 on dist-test-slave-h7jz"
server_key: "e92ea5eed5a092d15b2a5af0ab4717b9"
server_key_iv: "11b421b6312c84bd6c50e6f2d0558c0d"
server_key_version: "encryptionkey@0"
I20260618 22:33:20.644846 24297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:20.664976 24297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:20.665783 24297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:20.666139 24297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:20.682305 24297 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:40107
I20260618 22:33:20.682341 24362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:40107 every 8 connection(s)
I20260618 22:33:20.683534 24297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:20.685073 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 24297
I20260618 22:33:20.685261 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:20.686029 19034 external_mini_cluster.cc:1428] Setting key c3048fc4ff8ab8fb710070da816d3d93
I20260618 22:33:20.687811 24363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:20.698943 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:20.687769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.18.150.190:40107)
0618 22:33:20.688659 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:20.688670 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:20.688709 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:33:20.690203 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:20.690223 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:20.690684 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:20.691020 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:20.691557 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:20.692895 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:20.694699 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:20.695426 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:20.695578 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:20.696712 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:20.696720 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:20.696733 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:20.697279 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:20.697360 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:20.697380 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:20.697474 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:20.697681 (+ 207us) server_negotiation.cc:299] Negotiation successful
0618 22:33:20.698018 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":330,"threads_started":1}
I20260618 22:33:20.700150 24363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c: Bootstrap starting.
I20260618 22:33:20.702493 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:33:20.703227 24363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:20.704135 24363 log.cc:824] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:20.706920 24363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c: No bootstrap required, opened a new log
I20260618 22:33:20.713924 24363 raft_consensus.cc:348] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b584d5f61438495ac109bca99039c" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40107 } }
I20260618 22:33:20.714295 24363 raft_consensus.cc:374] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:20.714387 24363 raft_consensus.cc:729] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46b584d5f61438495ac109bca99039c, State: Initialized, Role: FOLLOWER
I20260618 22:33:20.714934 24363 consensus_queue.cc:260] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b584d5f61438495ac109bca99039c" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40107 } }
I20260618 22:33:20.715135 24363 raft_consensus.cc:388] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:20.715253 24363 raft_consensus.cc:482] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:20.715395 24363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:20.717381 24363 raft_consensus.cc:504] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b584d5f61438495ac109bca99039c" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40107 } }
I20260618 22:33:20.717839 24363 leader_election.cc:302] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b46b584d5f61438495ac109bca99039c; no voters:
I20260618 22:33:20.718354 24363 leader_election.cc:288] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:20.718487 24368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:20.719174 24368 raft_consensus.cc:686] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [term 1 LEADER]: Becoming Leader. State: Replica: b46b584d5f61438495ac109bca99039c, State: Running, Role: LEADER
I20260618 22:33:20.719655 24368 consensus_queue.cc:237] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b584d5f61438495ac109bca99039c" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40107 } }
I20260618 22:33:20.720309 24363 sys_catalog.cc:564] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:20.722302 24370 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b46b584d5f61438495ac109bca99039c. Latest consensus state: current_term: 1 leader_uuid: "b46b584d5f61438495ac109bca99039c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b584d5f61438495ac109bca99039c" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40107 } } }
I20260618 22:33:20.722669 24370 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:20.723296 24369 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b46b584d5f61438495ac109bca99039c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46b584d5f61438495ac109bca99039c" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40107 } } }
I20260618 22:33:20.723583 24369 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:20.726893 24376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:20.732013 24376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:20.740047 24376 catalog_manager.cc:1347] Generated new cluster ID: f3540627c4634f20b7ec0979c6810477
I20260618 22:33:20.740157 24376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:20.779016 24376 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:20.780499 24376 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:20.788429 24376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b46b584d5f61438495ac109bca99039c: Generated new TSK 0
I20260618 22:33:20.789273 24376 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260618 22:33:20.823839 24393 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260618 22:33:20.823946 24393 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260618 22:33:20.825292 24392 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:20.804689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:40107 (local address 127.0.0.1:42306)
0618 22:33:20.805347 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:20.805362 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:20.805475 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:20.805913 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:20.805916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:20.805924 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:33:20.806189 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:20.806196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:20.807559 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:20.807563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:20.808832 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:20.808846 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:20.809889 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:20.809894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:20.810079 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:20.810114 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:33:20.824812 (+ 14698us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40353/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0618 22:33:20.825010 (+ 198us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.150.190:40107: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40353/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":375,"threads_started":1}
I20260618 22:33:20.825289 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:20.804848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.18.150.190:40107)
0618 22:33:20.805040 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:20.805046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:20.805437 (+ 391us) server_negotiation.cc:407] Connection header received
0618 22:33:20.805648 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:20.805654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:20.805743 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:20.805916 (+ 173us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:33:20.806340 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:20.807440 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:20.808992 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:20.809684 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:20.809826 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:20.810401 (+ 575us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:33:20.824560 (+ 14159us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40353/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0618 22:33:20.824956 (+ 396us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42306: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40353/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":52}
W20260618 22:33:20.825400 24366 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42306: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40353/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260618 22:33:20.826781 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 24297
I20260618 22:33:20.827880 24358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:20.918639 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 24297
2026-06-18T22:33:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (714 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260618 22:33:20.969357 19034 mini_oidc.cc:152] Starting JWKS server
I20260618 22:33:20.971124 19034 webserver.cc:460] Webserver started at https://127.0.0.1:40513/ using document root <none> and password file <none>
I20260618 22:33:20.971331 19034 mini_oidc.cc:189] Starting OIDC Discovery server
I20260618 22:33:20.971887 19034 webserver.cc:460] Webserver started at http://127.0.0.1:46341/ using document root <none> and password file <none>
2026-06-18T22:33:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:20Z Disabled control of system clock
I20260618 22:33:20.994752 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:42995
--webserver_interface=127.18.150.190
--webserver_port=0
--builtin_ntp_servers=127.18.150.148:35521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:42995
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40513/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260618 22:33:21.295166 24400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:21.295450 24400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:21.295531 24400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:21.302583 24400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:21.302677 24400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:21.302721 24400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:21.302773 24400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:21.302814 24400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:33:21.302865 24400 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:33:21.302901 24400 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40513/jwks.json
I20260618 22:33:21.311668 24400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:42995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:42995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40513/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:21.312819 24400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:21.314512 24400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:21.325299 24405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:21.325335 24408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:21.325266 24406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:21.326670 24400 server_base.cc:1034] running on GCE node
I20260618 22:33:21.327445 24400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:21.328771 24400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:21.329988 24400 hybrid_clock.cc:648] HybridClock initialized: now 1781822001329909 us; error 64 us; skew 500 ppm
I20260618 22:33:21.330307 24400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:33:21.332305 24400 webserver.cc:460] Webserver started at http://127.18.150.190:38809/ using document root <none> and password file <none>
I20260618 22:33:21.332947 24400 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:21.333042 24400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:21.333340 24400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:21.336098 24400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "df147da2b33b4cfba0edc7874b06a777"
format_stamp: "Formatted at 2026-06-18 22:33:21 on dist-test-slave-h7jz"
server_key: "133ec39e2fdb67a2e8e19b3e8206b5ad"
server_key_iv: "eae9e3dc4c4b4141cc8a845ae91f77a0"
server_key_version: "encryptionkey@0"
I20260618 22:33:21.336831 24400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "df147da2b33b4cfba0edc7874b06a777"
format_stamp: "Formatted at 2026-06-18 22:33:21 on dist-test-slave-h7jz"
server_key: "133ec39e2fdb67a2e8e19b3e8206b5ad"
server_key_iv: "eae9e3dc4c4b4141cc8a845ae91f77a0"
server_key_version: "encryptionkey@0"
I20260618 22:33:21.341550 24400 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260618 22:33:21.344936 24414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:21.346282 24400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:33:21.346426 24400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "df147da2b33b4cfba0edc7874b06a777"
format_stamp: "Formatted at 2026-06-18 22:33:21 on dist-test-slave-h7jz"
server_key: "133ec39e2fdb67a2e8e19b3e8206b5ad"
server_key_iv: "eae9e3dc4c4b4141cc8a845ae91f77a0"
server_key_version: "encryptionkey@0"
I20260618 22:33:21.346607 24400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:21.379621 24400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:21.380467 24400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:21.380745 24400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:21.396780 24400 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:42995
I20260618 22:33:21.396798 24465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:42995 every 8 connection(s)
I20260618 22:33:21.398084 24400 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:21.400532 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 24400
I20260618 22:33:21.400765 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:21.401475 19034 external_mini_cluster.cc:1428] Setting key 3914e9b405f14d88c2cbb114a82c9f87
I20260618 22:33:21.402292 24466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:21.414860 24466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777: Bootstrap starting.
I20260618 22:33:21.415324 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:21.403370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55380 (local address 127.18.150.190:42995)
0618 22:33:21.404336 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:21.404347 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:21.404390 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:33:21.406625 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:21.406648 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:21.407123 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:21.407459 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:21.408000 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:21.409313 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:21.411031 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:21.411733 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:21.411867 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:21.413033 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:21.413041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:21.413055 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:21.413577 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:21.413651 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:21.413672 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:21.413763 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:21.414014 (+ 251us) server_negotiation.cc:299] Negotiation successful
0618 22:33:21.414337 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":344,"threads_started":1}
I20260618 22:33:21.417817 24466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:21.418758 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:33:21.418845 24466 log.cc:824] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:21.421723 24466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777: No bootstrap required, opened a new log
I20260618 22:33:21.428625 24466 raft_consensus.cc:348] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df147da2b33b4cfba0edc7874b06a777" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42995 } }
I20260618 22:33:21.428972 24466 raft_consensus.cc:374] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:21.429064 24466 raft_consensus.cc:729] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df147da2b33b4cfba0edc7874b06a777, State: Initialized, Role: FOLLOWER
I20260618 22:33:21.429571 24466 consensus_queue.cc:260] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df147da2b33b4cfba0edc7874b06a777" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42995 } }
I20260618 22:33:21.429774 24466 raft_consensus.cc:388] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:21.429886 24466 raft_consensus.cc:482] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:21.430060 24466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:21.432080 24466 raft_consensus.cc:504] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df147da2b33b4cfba0edc7874b06a777" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42995 } }
I20260618 22:33:21.432525 24466 leader_election.cc:302] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df147da2b33b4cfba0edc7874b06a777; no voters:
I20260618 22:33:21.433080 24466 leader_election.cc:288] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:21.433203 24471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:21.433930 24471 raft_consensus.cc:686] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [term 1 LEADER]: Becoming Leader. State: Replica: df147da2b33b4cfba0edc7874b06a777, State: Running, Role: LEADER
I20260618 22:33:21.434473 24471 consensus_queue.cc:237] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df147da2b33b4cfba0edc7874b06a777" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42995 } }
I20260618 22:33:21.435060 24466 sys_catalog.cc:564] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:21.437047 24473 sys_catalog.cc:455] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df147da2b33b4cfba0edc7874b06a777. Latest consensus state: current_term: 1 leader_uuid: "df147da2b33b4cfba0edc7874b06a777" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df147da2b33b4cfba0edc7874b06a777" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42995 } } }
I20260618 22:33:21.437386 24473 sys_catalog.cc:458] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:21.438097 24472 sys_catalog.cc:455] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df147da2b33b4cfba0edc7874b06a777" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df147da2b33b4cfba0edc7874b06a777" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42995 } } }
I20260618 22:33:21.438395 24472 sys_catalog.cc:458] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:21.442714 24476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:21.448237 24476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:21.456202 24476 catalog_manager.cc:1347] Generated new cluster ID: a8bcf04def20497c99e8ea962a7ed6e9
I20260618 22:33:21.456295 24476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:21.473215 24476 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:21.475018 24476 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:21.482462 24476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df147da2b33b4cfba0edc7874b06a777: Generated new TSK 0
I20260618 22:33:21.483403 24476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:33:21.507824 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:21.499938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55392 (local address 127.18.150.190:42995)
0618 22:33:21.500114 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:21.500121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:21.500460 (+ 339us) server_negotiation.cc:407] Connection header received
0618 22:33:21.500652 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:21.500657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:21.500750 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:21.500907 (+ 157us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:33:21.501285 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:21.502421 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:21.503720 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:21.504396 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:21.504515 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:21.504922 (+ 407us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:33:21.507204 (+ 2282us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40513/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0618 22:33:21.507518 (+ 314us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55392: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40513/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":69}
W20260618 22:33:21.507949 24469 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55392: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40513/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260618 22:33:21.509456 24495 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:21.499818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42995 (local address 127.0.0.1:55392)
0618 22:33:21.500364 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:21.500380 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:21.500493 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:21.500903 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:21.500905 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:21.500913 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:33:21.501156 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:21.501164 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:21.502536 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:21.502539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:21.503591 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:21.503602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:21.504546 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:21.504550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:21.504665 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:21.504695 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:33:21.507404 (+ 2709us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40513/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0618 22:33:21.509049 (+ 1645us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.150.190:42995: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40513/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":1384,"thread_start_us":281,"threads_started":1}
I20260618 22:33:21.510419 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 24400
I20260618 22:33:21.511567 24461 generic_service.cc:196] Checking for leaks (request via RPC)
E20260618 22:33:21.519341 24496 webserver.cc:533] Webserver: sslize failed:
E20260618 22:33:21.519438 24496 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260618 22:33:21.600891 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 24400
2026-06-18T22:33:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (682 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-18T22:33:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:21Z Disabled control of system clock
I20260618 22:33:21.673489 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35629
--webserver_interface=127.18.150.190
--webserver_port=0
--builtin_ntp_servers=127.18.150.148:41273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:35629
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/testchainca.cert with env {}
W20260618 22:33:21.977406 24501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:21.977692 24501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:21.977782 24501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:21.984871 24501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:21.984983 24501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:21.985033 24501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:21.985086 24501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:21.985126 24501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:33:21.985183 24501 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:33:21.985219 24501 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260618 22:33:21.995157 24501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:41273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:35629
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:21.996404 24501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:21.997632 24501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:22.008700 24506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:22.008649 24507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:22.009006 24509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:22.009305 24501 server_base.cc:1034] running on GCE node
I20260618 22:33:22.010121 24501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:22.011467 24501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:22.012674 24501 hybrid_clock.cc:648] HybridClock initialized: now 1781822002012615 us; error 53 us; skew 500 ppm
I20260618 22:33:22.013005 24501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:33:22.015185 24501 webserver.cc:460] Webserver started at http://127.18.150.190:44109/ using document root <none> and password file <none>
I20260618 22:33:22.015851 24501 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:22.015947 24501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:22.016240 24501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:22.019100 24501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "96356c9a82ee4f2a852780430179fe37"
format_stamp: "Formatted at 2026-06-18 22:33:22 on dist-test-slave-h7jz"
server_key: "2e3ec698686626525358d8584dcf97d1"
server_key_iv: "6e6e449c567b96843584bdbfef65b879"
server_key_version: "encryptionkey@0"
I20260618 22:33:22.019855 24501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "96356c9a82ee4f2a852780430179fe37"
format_stamp: "Formatted at 2026-06-18 22:33:22 on dist-test-slave-h7jz"
server_key: "2e3ec698686626525358d8584dcf97d1"
server_key_iv: "6e6e449c567b96843584bdbfef65b879"
server_key_version: "encryptionkey@0"
I20260618 22:33:22.024716 24501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:33:22.028007 24515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:22.029284 24501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:33:22.029421 24501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "96356c9a82ee4f2a852780430179fe37"
format_stamp: "Formatted at 2026-06-18 22:33:22 on dist-test-slave-h7jz"
server_key: "2e3ec698686626525358d8584dcf97d1"
server_key_iv: "6e6e449c567b96843584bdbfef65b879"
server_key_version: "encryptionkey@0"
I20260618 22:33:22.029606 24501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:22.042127 24501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:22.042901 24501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:22.043145 24501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:22.059145 24501 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:35629
I20260618 22:33:22.059150 24566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:35629 every 8 connection(s)
I20260618 22:33:22.060335 24501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:22.063697 24567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:22.068996 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 24501
I20260618 22:33:22.069175 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:22.069986 19034 external_mini_cluster.cc:1428] Setting key 0414ecb2424c0c787972f27267e5bdfb
I20260618 22:33:22.074339 24567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37: Bootstrap starting.
I20260618 22:33:22.077466 24567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:22.078658 24567 log.cc:824] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:22.081672 24567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37: No bootstrap required, opened a new log
I20260618 22:33:22.083216 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:22.071868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50712 (local address 127.18.150.190:35629)
0618 22:33:22.072819 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:22.072829 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:22.072867 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:33:22.074439 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:22.074466 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:22.074931 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:22.075240 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:22.075832 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:22.077041 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:22.078812 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:22.079792 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:22.079982 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:22.081030 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:22.081037 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:22.081050 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:33:22.081541 (+ 491us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:22.081612 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:33:22.081632 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:22.081732 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:22.081960 (+ 228us) server_negotiation.cc:299] Negotiation successful
0618 22:33:22.082278 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":337,"threads_started":1}
I20260618 22:33:22.086743 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:33:22.090029 24567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96356c9a82ee4f2a852780430179fe37" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35629 } }
I20260618 22:33:22.090370 24567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:22.090462 24567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96356c9a82ee4f2a852780430179fe37, State: Initialized, Role: FOLLOWER
I20260618 22:33:22.090943 24567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96356c9a82ee4f2a852780430179fe37" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35629 } }
I20260618 22:33:22.091138 24567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:22.091256 24567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:22.091395 24567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:22.093374 24567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96356c9a82ee4f2a852780430179fe37" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35629 } }
I20260618 22:33:22.093811 24567 leader_election.cc:302] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96356c9a82ee4f2a852780430179fe37; no voters:
I20260618 22:33:22.094348 24567 leader_election.cc:288] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:22.094439 24572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:22.095065 24572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [term 1 LEADER]: Becoming Leader. State: Replica: 96356c9a82ee4f2a852780430179fe37, State: Running, Role: LEADER
I20260618 22:33:22.095570 24572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96356c9a82ee4f2a852780430179fe37" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35629 } }
I20260618 22:33:22.096278 24567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:22.098348 24573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96356c9a82ee4f2a852780430179fe37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96356c9a82ee4f2a852780430179fe37" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35629 } } }
I20260618 22:33:22.098520 24574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96356c9a82ee4f2a852780430179fe37. Latest consensus state: current_term: 1 leader_uuid: "96356c9a82ee4f2a852780430179fe37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96356c9a82ee4f2a852780430179fe37" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35629 } } }
I20260618 22:33:22.098718 24573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:22.098793 24574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:22.100055 24581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:22.104713 24581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:22.113140 24581 catalog_manager.cc:1347] Generated new cluster ID: 3fd78abdcade4a04b696f24efa4a2c97
I20260618 22:33:22.113253 24581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:22.126947 24581 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:22.128666 24581 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:22.137466 24581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96356c9a82ee4f2a852780430179fe37: Generated new TSK 0
I20260618 22:33:22.138479 24581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:33:22.673851 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:22.153642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50720 (local address 127.18.150.190:35629)
0618 22:33:22.153809 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:22.153815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:22.154206 (+ 391us) server_negotiation.cc:407] Connection header received
0618 22:33:22.154418 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:22.154421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:22.154492 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:22.154609 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:33:22.155088 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:22.155754 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:22.157699 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:22.158154 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:22.158250 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:22.158751 (+ 501us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:33:22.673125 (+514374us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0618 22:33:22.673542 (+ 417us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50720: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":44}
W20260618 22:33:22.674006 24570 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50720: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260618 22:33:22.674180 24596 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:22.153489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35629 (local address 127.0.0.1:50720)
0618 22:33:22.154075 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:22.154094 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:22.154252 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:22.154649 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:22.154654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:22.154666 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:33:22.154947 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:22.154959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:22.155884 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:22.155890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:22.157573 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:22.157588 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:22.158305 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:22.158313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:22.158495 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:22.158538 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:33:22.673453 (+514915us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0618 22:33:22.673775 (+ 322us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.150.190:35629: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":278,"threads_started":1}
I20260618 22:33:22.675271 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 24501
I20260618 22:33:22.676275 24562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:22.775367 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 24501
2026-06-18T22:33:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1155 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (391 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (385 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:33:23 dist-test-slave-h7jz krb5kdc[24611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:33:23 dist-test-slave-h7jz krb5kdc[24611](info): set up 2 sockets
Jun 18 22:33:23 dist-test-slave-h7jz krb5kdc[24611](info): commencing operation
krb5kdc: starting...
W20260618 22:33:24.550242 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:33:24 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:33:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:33:24.735109 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:44801
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:44801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:25.038883 24624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:25.039170 24624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:25.039297 24624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:25.046306 24624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:25.046417 24624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:25.046494 24624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:25.046569 24624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:25.046636 24624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:25.056082 24624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:44801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:44801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:25.057325 24624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:25.058579 24624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:25.069540 24632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:25.069609 24629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:25.069589 24630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:25.070247 24624 server_base.cc:1034] running on GCE node
I20260618 22:33:25.070928 24624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:25.072180 24624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:25.073385 24624 hybrid_clock.cc:648] HybridClock initialized: now 1781822005073329 us; error 46 us; skew 500 ppm
I20260618 22:33:25.073695 24624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:25 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822005, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:25.079399 24624 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:33:25.080709 24624 webserver.cc:460] Webserver started at http://127.18.150.190:39255/ using document root <none> and password file <none>
I20260618 22:33:25.081313 24624 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:25.081403 24624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:25.081645 24624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:25.084524 24624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "c53654a73c3e4433beac1f24e521abba"
format_stamp: "Formatted at 2026-06-18 22:33:25 on dist-test-slave-h7jz"
server_key: "cb61456735376e2609b0e904c82f4ab6"
server_key_iv: "ce1d1eff980f57cdfb85ca346e5da281"
server_key_version: "encryptionkey@0"
I20260618 22:33:25.085286 24624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "c53654a73c3e4433beac1f24e521abba"
format_stamp: "Formatted at 2026-06-18 22:33:25 on dist-test-slave-h7jz"
server_key: "cb61456735376e2609b0e904c82f4ab6"
server_key_iv: "ce1d1eff980f57cdfb85ca346e5da281"
server_key_version: "encryptionkey@0"
I20260618 22:33:25.090214 24624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:33:25.093580 24639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:25.094903 24624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:25.095052 24624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "c53654a73c3e4433beac1f24e521abba"
format_stamp: "Formatted at 2026-06-18 22:33:25 on dist-test-slave-h7jz"
server_key: "cb61456735376e2609b0e904c82f4ab6"
server_key_iv: "ce1d1eff980f57cdfb85ca346e5da281"
server_key_version: "encryptionkey@0"
I20260618 22:33:25.095230 24624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:25.108170 24624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:25.111948 24624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:25.112193 24624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:25.128257 24624 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:44801
I20260618 22:33:25.128290 24690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:44801 every 8 connection(s)
I20260618 22:33:25.129585 24624 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:25.131740 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 24624
I20260618 22:33:25.131910 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:25.132493 19034 external_mini_cluster.cc:1428] Setting key e14b6f4d1f1d440c239ac32ee205609c
I20260618 22:33:25.133196 24691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 18 22:33:25 dist-test-slave-h7jz krb5kdc[24611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:25.146543 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba: Bootstrap starting.
I20260618 22:33:25.150177 24691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:25.151327 24691 log.cc:824] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:25.154449 24691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba: No bootstrap required, opened a new log
I20260618 22:33:25.154973 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:25.134337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.18.150.190:44801)
0618 22:33:25.135004 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:25.135015 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:25.135052 (+ 37us) server_negotiation.cc:407] Connection header received
0618 22:33:25.136492 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:25.136512 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:25.136885 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:25.137215 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:25.138877 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:25.140287 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:25.141884 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:25.142637 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:25.142781 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:25.147200 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:25.147210 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:25.147238 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:25.147279 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:25.150852 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:25.151497 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:25.151505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:25.151512 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:25.151705 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:25.152110 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:25.152114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:25.152116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:25.152677 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:25.152938 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:25.153295 (+ 357us) server_negotiation.cc:299] Negotiation successful
0618 22:33:25.153694 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":291,"threads_started":1}
I20260618 22:33:25.161810 24691 raft_consensus.cc:348] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53654a73c3e4433beac1f24e521abba" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 44801 } }
I20260618 22:33:25.162158 24691 raft_consensus.cc:374] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:25.162231 24691 raft_consensus.cc:729] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c53654a73c3e4433beac1f24e521abba, State: Initialized, Role: FOLLOWER
I20260618 22:33:25.162760 24691 consensus_queue.cc:260] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53654a73c3e4433beac1f24e521abba" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 44801 } }
I20260618 22:33:25.162949 24691 raft_consensus.cc:388] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:25.163040 24691 raft_consensus.cc:482] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:25.163154 24691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:25.165191 24691 raft_consensus.cc:504] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53654a73c3e4433beac1f24e521abba" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 44801 } }
I20260618 22:33:25.165683 24691 leader_election.cc:302] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c53654a73c3e4433beac1f24e521abba; no voters:
I20260618 22:33:25.166301 24691 leader_election.cc:288] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:25.166381 24696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:25.166937 24696 raft_consensus.cc:686] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [term 1 LEADER]: Becoming Leader. State: Replica: c53654a73c3e4433beac1f24e521abba, State: Running, Role: LEADER
I20260618 22:33:25.167434 24696 consensus_queue.cc:237] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53654a73c3e4433beac1f24e521abba" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 44801 } }
I20260618 22:33:25.168094 24691 sys_catalog.cc:564] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:25.170286 24697 sys_catalog.cc:455] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c53654a73c3e4433beac1f24e521abba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53654a73c3e4433beac1f24e521abba" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 44801 } } }
I20260618 22:33:25.170656 24697 sys_catalog.cc:458] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:25.171068 24698 sys_catalog.cc:455] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [sys.catalog]: SysCatalogTable state changed. Reason: New leader c53654a73c3e4433beac1f24e521abba. Latest consensus state: current_term: 1 leader_uuid: "c53654a73c3e4433beac1f24e521abba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53654a73c3e4433beac1f24e521abba" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 44801 } } }
I20260618 22:33:25.171347 24698 sys_catalog.cc:458] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:25.171974 24704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:25.175868 24704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:25.184038 24704 catalog_manager.cc:1347] Generated new cluster ID: 36ffc52ef0e54baf996d374944d7260a
I20260618 22:33:25.184154 24704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:25.197079 24704 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:25.198415 24704 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:25.207018 24704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c53654a73c3e4433beac1f24e521abba: Generated new TSK 0
I20260618 22:33:25.207777 24704 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:33:25.261866 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:44801
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:25.659044 24717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:25.659422 24717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:25.659571 24717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:25.671504 24717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:25.671655 24717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:25.671895 24717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:33:25.687963 24717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:44801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:25.689738 24717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:25.691485 24717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:25.705874 24723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:25.706043 24722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:25.706480 24725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:25.706564 24717 server_base.cc:1034] running on GCE node
I20260618 22:33:25.707293 24717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:25.708410 24717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:25.709648 24717 hybrid_clock.cc:648] HybridClock initialized: now 1781822005709584 us; error 44 us; skew 500 ppm
I20260618 22:33:25.710057 24717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:25 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822005, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:25.713914 24717 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:33:25.715610 24717 webserver.cc:460] Webserver started at http://127.18.150.129:39835/ using document root <none> and password file <none>
I20260618 22:33:25.716490 24717 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:25.716619 24717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:25.716969 24717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:25.721012 24717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "6e6746b8ded142e388b65da19e240de7"
format_stamp: "Formatted at 2026-06-18 22:33:25 on dist-test-slave-h7jz"
server_key: "c89aa3821b47da63b6f287bd4189c810"
server_key_iv: "ccf96ea921e993e9a261255181df2437"
server_key_version: "encryptionkey@0"
I20260618 22:33:25.722023 24717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "6e6746b8ded142e388b65da19e240de7"
format_stamp: "Formatted at 2026-06-18 22:33:25 on dist-test-slave-h7jz"
server_key: "c89aa3821b47da63b6f287bd4189c810"
server_key_iv: "ccf96ea921e993e9a261255181df2437"
server_key_version: "encryptionkey@0"
I20260618 22:33:25.729254 24717 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.004s
I20260618 22:33:25.733992 24732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:25.735711 24717 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:33:25.735886 24717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "6e6746b8ded142e388b65da19e240de7"
format_stamp: "Formatted at 2026-06-18 22:33:25 on dist-test-slave-h7jz"
server_key: "c89aa3821b47da63b6f287bd4189c810"
server_key_iv: "ccf96ea921e993e9a261255181df2437"
server_key_version: "encryptionkey@0"
I20260618 22:33:25.736121 24717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:25.760217 24717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:25.765588 24717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:25.765853 24717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:25.766826 24717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:25.768178 24717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:25.768249 24717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:25.768353 24717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:25.768397 24717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:25.797461 24717 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:34007
I20260618 22:33:25.797513 24844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:34007 every 8 connection(s)
I20260618 22:33:25.798734 24717 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:33:25.805475 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 24717
I20260618 22:33:25.805642 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:33:25.806377 19034 external_mini_cluster.cc:1428] Setting key e2b089a8316df0499cd8ad976ba3e23a
Jun 18 22:33:25 dist-test-slave-h7jz krb5kdc[24611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822005, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:25.819023 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:25.801158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:56143 (local address 127.18.150.190:44801)
0618 22:33:25.801576 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:25.801582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:25.802599 (+ 1017us) server_negotiation.cc:407] Connection header received
0618 22:33:25.804339 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:25.804343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:25.804413 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:25.804505 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:25.806017 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:25.807034 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:25.808926 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:25.809490 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:25.809611 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:25.813613 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:25.813620 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:25.813625 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:25.813663 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:25.816169 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:25.816975 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:25.816979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:25.816981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:25.817087 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:25.817481 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:25.817484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:25.817486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:25.817715 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:25.817816 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:25.818613 (+ 797us) server_negotiation.cc:299] Negotiation successful
0618 22:33:25.818778 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":191,"threads_started":1}
I20260618 22:33:25.820374 24848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:25.801739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:44801 (local address 127.18.150.129:56143)
0618 22:33:25.802454 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:25.802491 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:25.804057 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:25.804727 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:25.804735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:25.805219 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:25.805834 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:25.805853 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:25.807195 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:25.807199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:25.808766 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:25.808782 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:25.809844 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:25.809854 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:25.810329 (+ 475us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:25.811173 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:25.811200 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:25.813388 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:25.816298 (+ 2910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:25.816307 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:25.816324 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:25.816881 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:25.817170 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:25.817175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:25.817179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:25.817396 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:25.817820 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:25.817827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:25.818274 (+ 447us) client_negotiation.cc:769] Sending connection context
0618 22:33:25.818555 (+ 281us) client_negotiation.cc:240] Negotiation successful
0618 22:33:25.819066 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":296,"threads_started":1}
I20260618 22:33:25.821899 24845 heartbeater.cc:347] Connected to a master server at 127.18.150.190:44801
I20260618 22:33:25.822201 24845 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:25.822878 24845 heartbeater.cc:511] Master 127.18.150.190:44801 requested a full tablet report, sending...
I20260618 22:33:25.824934 24656 ts_manager.cc:194] Registered new tserver with Master: 6e6746b8ded142e388b65da19e240de7 (127.18.150.129:34007)
I20260618 22:33:25.826395 24656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:56143
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:33:25.851339 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:44801
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:26.186173 24851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:26.186508 24851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:26.186674 24851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:26.194062 24851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:26.194208 24851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:26.194373 24851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:33:26.204097 24851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:44801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:26.205400 24851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:26.206853 24851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:26.219363 24857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:26.219417 24859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:26.219383 24856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:26.220268 24851 server_base.cc:1034] running on GCE node
I20260618 22:33:26.220829 24851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:26.221728 24851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:26.222924 24851 hybrid_clock.cc:648] HybridClock initialized: now 1781822006222858 us; error 54 us; skew 500 ppm
I20260618 22:33:26.223223 24851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:26.226225 24851 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:33:26.227560 24851 webserver.cc:460] Webserver started at http://127.18.150.130:42523/ using document root <none> and password file <none>
I20260618 22:33:26.228202 24851 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:26.228315 24851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:26.228597 24851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:26.231426 24851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "0a74ac75942c40e8b18dd38eae1ba0ce"
format_stamp: "Formatted at 2026-06-18 22:33:26 on dist-test-slave-h7jz"
server_key: "9a0ea68f7e8061c28df96f84b12dbb2e"
server_key_iv: "e5fc48e06086ca5e86d88d6e24bf9a61"
server_key_version: "encryptionkey@0"
I20260618 22:33:26.232136 24851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "0a74ac75942c40e8b18dd38eae1ba0ce"
format_stamp: "Formatted at 2026-06-18 22:33:26 on dist-test-slave-h7jz"
server_key: "9a0ea68f7e8061c28df96f84b12dbb2e"
server_key_iv: "e5fc48e06086ca5e86d88d6e24bf9a61"
server_key_version: "encryptionkey@0"
I20260618 22:33:26.236882 24851 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260618 22:33:26.240268 24866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:26.241573 24851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260618 22:33:26.241706 24851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "0a74ac75942c40e8b18dd38eae1ba0ce"
format_stamp: "Formatted at 2026-06-18 22:33:26 on dist-test-slave-h7jz"
server_key: "9a0ea68f7e8061c28df96f84b12dbb2e"
server_key_iv: "e5fc48e06086ca5e86d88d6e24bf9a61"
server_key_version: "encryptionkey@0"
I20260618 22:33:26.241884 24851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:26.267414 24851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:26.271384 24851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:26.271627 24851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:26.272517 24851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:26.273756 24851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:26.273828 24851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:26.273957 24851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:26.274019 24851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260618 22:33:26.302168 24851 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:32979
I20260618 22:33:26.302220 24978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:32979 every 8 connection(s)
I20260618 22:33:26.303361 24851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:33:26.307870 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 24851
I20260618 22:33:26.308002 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:33:26.308413 19034 external_mini_cluster.cc:1428] Setting key b0248ca554aa4be8a7d345ae9b079104
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:26.322469 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:26.305726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:33551 (local address 127.18.150.190:44801)
0618 22:33:26.305911 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:26.305916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:26.307085 (+ 1169us) server_negotiation.cc:407] Connection header received
0618 22:33:26.308887 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:26.308891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:26.308957 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:26.309037 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:26.310807 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.311784 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.313257 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.313813 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.313924 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:26.317109 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:26.317113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:26.317117 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:26.317147 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:26.319537 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.320361 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.320365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.320367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.320484 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.320873 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.320876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.320878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.321107 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:26.321237 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:26.322050 (+ 813us) server_negotiation.cc:299] Negotiation successful
0618 22:33:26.322225 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:33:26.323853 24981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:26.306268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:44801 (local address 127.18.150.130:33551)
0618 22:33:26.306929 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:26.306973 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:26.308630 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:26.309279 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:26.309289 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:26.309981 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:26.310650 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:26.310667 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.311919 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.311923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:26.313124 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:26.313136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.314011 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.314016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:26.314226 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:26.314827 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:26.314848 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:26.316906 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:26.319657 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:26.319666 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:26.319684 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:26.320252 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:26.320567 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:26.320571 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:26.320575 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:26.320785 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:26.321231 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:26.321239 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:26.321642 (+ 403us) client_negotiation.cc:769] Sending connection context
0618 22:33:26.321926 (+ 284us) client_negotiation.cc:240] Negotiation successful
0618 22:33:26.322494 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":249,"threads_started":1}
I20260618 22:33:26.325220 24979 heartbeater.cc:347] Connected to a master server at 127.18.150.190:44801
I20260618 22:33:26.325548 24979 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:26.326437 24979 heartbeater.cc:511] Master 127.18.150.190:44801 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:33:26.328069 24656 ts_manager.cc:194] Registered new tserver with Master: 0a74ac75942c40e8b18dd38eae1ba0ce (127.18.150.130:32979)
I20260618 22:33:26.328884 24656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:33551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:33:26.353578 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:44801
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:26.670215 24984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:26.670486 24984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:26.670581 24984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:26.677807 24984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:26.677920 24984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:26.678058 24984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:33:26.690078 24984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:44801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:26.691341 24984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:26.692602 24984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:26.705090 24990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:26.705127 24989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:26.705053 24992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:26.706095 24984 server_base.cc:1034] running on GCE node
I20260618 22:33:26.706714 24984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:26.707603 24984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:26.708807 24984 hybrid_clock.cc:648] HybridClock initialized: now 1781822006708757 us; error 43 us; skew 500 ppm
I20260618 22:33:26.709095 24984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:26.712034 24984 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:33:26.713428 24984 webserver.cc:460] Webserver started at http://127.18.150.131:43547/ using document root <none> and password file <none>
I20260618 22:33:26.714110 24984 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:26.714227 24984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:26.714502 24984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:26.717284 24984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "41d38dc848ad4ce6b8a19d3762f4a9c4"
format_stamp: "Formatted at 2026-06-18 22:33:26 on dist-test-slave-h7jz"
server_key: "8f282b00f6d205698fe996e3aba64837"
server_key_iv: "ccb51459fce1f5d044608aef8045109f"
server_key_version: "encryptionkey@0"
I20260618 22:33:26.718044 24984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "41d38dc848ad4ce6b8a19d3762f4a9c4"
format_stamp: "Formatted at 2026-06-18 22:33:26 on dist-test-slave-h7jz"
server_key: "8f282b00f6d205698fe996e3aba64837"
server_key_iv: "ccb51459fce1f5d044608aef8045109f"
server_key_version: "encryptionkey@0"
I20260618 22:33:26.722995 24984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:33:26.726397 24999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:26.727700 24984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:33:26.727842 24984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "41d38dc848ad4ce6b8a19d3762f4a9c4"
format_stamp: "Formatted at 2026-06-18 22:33:26 on dist-test-slave-h7jz"
server_key: "8f282b00f6d205698fe996e3aba64837"
server_key_iv: "ccb51459fce1f5d044608aef8045109f"
server_key_version: "encryptionkey@0"
I20260618 22:33:26.728016 24984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:26.758605 24984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:26.763131 24984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:26.763415 24984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:26.764427 24984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:26.765897 24984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:26.765996 24984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260618 22:33:26.766119 24984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:26.766187 24984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:26.795326 24984 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:40307
I20260618 22:33:26.795372 25111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:40307 every 8 connection(s)
I20260618 22:33:26.796569 24984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:33:26.801220 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 24984
I20260618 22:33:26.801388 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:33:26.801924 19034 external_mini_cluster.cc:1428] Setting key a502012adcf82f43a5c3bcc9818c621d
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:26.817523 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:26.801141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:32889 (local address 127.18.150.190:44801)
0618 22:33:26.801320 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:26.801325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:26.801348 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:33:26.802676 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:26.802682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:26.802783 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:26.802897 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:26.804740 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.805572 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.807152 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.807554 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.807665 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:26.810966 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:26.810978 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:26.810981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:26.811016 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:26.814310 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.815160 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.815165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.815169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.815313 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.815774 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.815779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.815782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.816123 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:26.816255 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:26.816970 (+ 715us) server_negotiation.cc:299] Negotiation successful
0618 22:33:26.817181 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260618 22:33:26.818558 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:26.799640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:44801 (local address 127.18.150.131:32889)
0618 22:33:26.800402 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:26.800462 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:26.802344 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:26.803141 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:26.803153 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:26.803766 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:26.804540 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:26.804577 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.805717 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.805721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:26.807013 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:26.807025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.807787 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.807793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:26.807996 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:26.808628 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:26.808650 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:26.810758 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:26.814462 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:26.814470 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:26.814483 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:26.815026 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:26.815413 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:26.815418 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:26.815422 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:26.815665 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:26.816233 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:26.816241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:26.816585 (+ 344us) client_negotiation.cc:769] Sending connection context
0618 22:33:26.816835 (+ 250us) client_negotiation.cc:240] Negotiation successful
0618 22:33:26.817400 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":324,"threads_started":1}
I20260618 22:33:26.819805 25112 heartbeater.cc:347] Connected to a master server at 127.18.150.190:44801
I20260618 22:33:26.820083 25112 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:26.820778 25112 heartbeater.cc:511] Master 127.18.150.190:44801 requested a full tablet report, sending...
I20260618 22:33:26.822121 24656 ts_manager.cc:194] Registered new tserver with Master: 41d38dc848ad4ce6b8a19d3762f4a9c4 (127.18.150.131:40307)
I20260618 22:33:26.822876 24656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:32889
I20260618 22:33:26.829209 24845 heartbeater.cc:503] Master 127.18.150.190:44801 was elected leader, sending a full tablet report...
I20260618 22:33:26.829255 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/test-admin.keytab.
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:26.855338 19034 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260618 22:33:26.859880 25119 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:26.857758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.18.150.129:34007)
0618 22:33:26.858333 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:26.858339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:26.858365 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:33:26.858426 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:26.858432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:26.858635 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:26.858780 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:26.859505 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54376: BlockingRecv error: recv got EOF from 127.0.0.1:54376 (error 108)
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":281,"threads_started":1}
W20260618 22:33:26.859920 19034 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/krb5cc' of type FILE
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:26.861603 19034 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:26.862865 19034 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:33:26.877718 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:26.864368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54382 (local address 127.18.150.129:34007)
0618 22:33:26.864529 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:26.864535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:26.864658 (+ 123us) server_negotiation.cc:407] Connection header received
0618 22:33:26.864803 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:26.864806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:26.864859 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:26.864956 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:26.865872 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.866939 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.868091 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.868683 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.868832 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:26.872517 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:26.872530 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:26.872546 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:26.872582 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:26.875325 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.875951 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.875957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.875959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.876066 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.876416 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.876419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.876421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.876757 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:26.877001 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:26.877194 (+ 193us) server_negotiation.cc:299] Negotiation successful
0618 22:33:26.877432 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260618 22:33:26.883345 25119 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:26.881650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54388 (local address 127.18.150.129:34007)
0618 22:33:26.881802 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:26.881806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:26.882197 (+ 391us) server_negotiation.cc:407] Connection header received
0618 22:33:26.882344 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:26.882348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:26.882418 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:26.882493 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:26.883115 (+ 622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54388: BlockingRecv error: recv got EOF from 127.0.0.1:54388 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260618 22:33:26.883399 19034 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/krb5cc' of type FILE
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:26.884799 19034 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:26.885994 19034 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:33:26.899468 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:26.887298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.18.150.129:34007)
0618 22:33:26.887474 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:26.887480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:26.887772 (+ 292us) server_negotiation.cc:407] Connection header received
0618 22:33:26.888065 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:26.888069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:26.888125 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:26.888220 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:26.889146 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.889828 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.890987 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.891407 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.891524 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:26.894802 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:26.894806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:26.894810 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:26.894841 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:26.897056 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.897659 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.897664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.897673 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.897787 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.898170 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.898175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.898177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.898458 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:26.898696 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:26.898960 (+ 264us) server_negotiation.cc:299] Negotiation successful
0618 22:33:26.899145 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260618 22:33:26.903504 25119 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:26.902003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54406 (local address 127.18.150.129:34007)
0618 22:33:26.902172 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:26.902177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:26.902399 (+ 222us) server_negotiation.cc:407] Connection header received
0618 22:33:26.902538 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:26.902543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:26.902608 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:26.902681 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:26.903312 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54406: BlockingRecv error: recv got EOF from 127.0.0.1:54406 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260618 22:33:26.903899 19034 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821960891835-19034-0/krb5kdc/krb5cc' of type FILE
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:26.905238 19034 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:26.906441 19034 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 18 22:33:26 dist-test-slave-h7jz krb5kdc[24611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:33:26.919467 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:26.907591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.18.150.129:34007)
0618 22:33:26.907740 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:26.907746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:26.908027 (+ 281us) server_negotiation.cc:407] Connection header received
0618 22:33:26.908187 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:26.908191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:26.908245 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:26.908320 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:26.909231 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.909920 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.911088 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.911496 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.911597 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:26.914772 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:26.914777 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:26.914780 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:26.914811 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:26.917173 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.917738 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.917742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.917744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.917870 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.918318 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.918321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.918323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.918588 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:26.918856 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:26.919047 (+ 191us) server_negotiation.cc:299] Negotiation successful
0618 22:33:26.919208 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:33:26.920872 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 24717
I20260618 22:33:26.931459 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:26.921478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.18.150.129:34007)
0618 22:33:26.921633 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:26.921650 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:26.921957 (+ 307us) server_negotiation.cc:407] Connection header received
0618 22:33:26.922080 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:26.922084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:26.922136 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:26.922211 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:26.923014 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.923679 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.924833 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:26.925231 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:26.925327 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:26.927113 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:26.927118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:26.927121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:26.927148 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:26.929273 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.929779 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.929783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.929785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.929894 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:26.930230 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:26.930234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:26.930236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:26.930482 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:26.930699 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:26.930931 (+ 232us) server_negotiation.cc:299] Negotiation successful
0618 22:33:26.931103 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:33:26.932025 24840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:27.015853 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 24717
I20260618 22:33:27.028265 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 24851
Jun 18 22:33:27 dist-test-slave-h7jz krb5kdc[24611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:33:27.042647 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:27.029103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.18.150.130:32979)
0618 22:33:27.029621 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:27.029628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:27.029651 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:33:27.029710 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:27.029716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:27.029924 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:27.030097 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:27.031031 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:27.032064 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:27.033289 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:27.033824 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:27.034045 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:27.037435 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:27.037446 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:27.037461 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:27.037496 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:27.040042 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:27.040662 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:27.040669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:27.040672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:27.040789 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:27.041130 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:27.041135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:27.041137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:27.041478 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:27.041756 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:27.042099 (+ 343us) server_negotiation.cc:299] Negotiation successful
0618 22:33:27.042354 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":267,"threads_started":1}
I20260618 22:33:27.043859 24974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:27.123591 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 24851
I20260618 22:33:27.135664 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 24984
Jun 18 22:33:27 dist-test-slave-h7jz krb5kdc[24611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:33:27.150740 25134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:27.136626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57904 (local address 127.18.150.131:40307)
0618 22:33:27.137298 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:27.137306 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:27.137333 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:33:27.137388 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:27.137394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:27.137597 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:27.137731 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:27.138703 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:27.139732 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:27.140950 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:27.141499 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:27.141601 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:27.145511 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:27.145526 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:27.145541 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:27.145577 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:27.148266 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:27.148969 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:27.148977 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:27.148979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:27.149091 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:27.149390 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:27.149393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:27.149395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:27.149707 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:27.149998 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:27.150279 (+ 281us) server_negotiation.cc:299] Negotiation successful
0618 22:33:27.150471 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":377,"threads_started":1}
I20260618 22:33:27.151997 25107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:27.232923 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 24984
I20260618 22:33:27.244904 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 24624
I20260618 22:33:27.246119 24686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:27.345911 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 24624
2026-06-18T22:33:27Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3796 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:33:27 dist-test-slave-h7jz krb5kdc[25139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:33:27 dist-test-slave-h7jz krb5kdc[25139](info): set up 2 sockets
Jun 18 22:33:27 dist-test-slave-h7jz krb5kdc[25139](info): commencing operation
krb5kdc: starting...
W20260618 22:33:28.346428 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.005s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:33:28 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:33:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:28Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.190.keytab.
Entry for principal oryx/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.190.keytab.
I20260618 22:33:28.528858 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35785
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.190.keytab
--principal=oryx/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:35785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:28.840024 25152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:28.840317 25152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:28.840440 25152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:28.847707 25152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:28.847826 25152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:28.847896 25152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:28.847973 25152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:28.848030 25152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:28.857822 25152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:35785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.190.keytab
--principal=oryx/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:28.859218 25152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:28.860497 25152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:28.871541 25160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:28.871649 25157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:28.871619 25158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:28.872506 25152 server_base.cc:1034] running on GCE node
I20260618 22:33:28.873257 25152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:28.874688 25152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:28.875910 25152 hybrid_clock.cc:648] HybridClock initialized: now 1781822008875859 us; error 39 us; skew 500 ppm
I20260618 22:33:28.876238 25152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:28 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822008, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:28.879245 25152 init.cc:377] Logged in from keytab as oryx/127.18.150.190@KRBTEST.COM (short username oryx)
I20260618 22:33:28.880723 25152 webserver.cc:460] Webserver started at http://127.18.150.190:45129/ using document root <none> and password file <none>
I20260618 22:33:28.881376 25152 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:28.881489 25152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:28.881767 25152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:28.884631 25152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "2b9615ea036b408b8b1da7b8b15b4263"
format_stamp: "Formatted at 2026-06-18 22:33:28 on dist-test-slave-h7jz"
server_key: "0809d480a1efaa3913b2f6c988a1557e"
server_key_iv: "3a6b3e7bf455b8ee9a577e3341a16b5a"
server_key_version: "encryptionkey@0"
I20260618 22:33:28.885332 25152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "2b9615ea036b408b8b1da7b8b15b4263"
format_stamp: "Formatted at 2026-06-18 22:33:28 on dist-test-slave-h7jz"
server_key: "0809d480a1efaa3913b2f6c988a1557e"
server_key_iv: "3a6b3e7bf455b8ee9a577e3341a16b5a"
server_key_version: "encryptionkey@0"
I20260618 22:33:28.890264 25152 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260618 22:33:28.893388 25167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:28.894718 25152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:28.894866 25152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "2b9615ea036b408b8b1da7b8b15b4263"
format_stamp: "Formatted at 2026-06-18 22:33:28 on dist-test-slave-h7jz"
server_key: "0809d480a1efaa3913b2f6c988a1557e"
server_key_iv: "3a6b3e7bf455b8ee9a577e3341a16b5a"
server_key_version: "encryptionkey@0"
I20260618 22:33:28.895036 25152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:28.909683 25152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:28.914080 25152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:28.914369 25152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:28.930841 25152 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:35785
I20260618 22:33:28.930863 25218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:35785 every 8 connection(s)
I20260618 22:33:28.932150 25152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:28.935676 25219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:28.936614 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 25152
I20260618 22:33:28.936829 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:28.937592 19034 external_mini_cluster.cc:1428] Setting key 2223feaa8bc580133998dce3a28b7f54
I20260618 22:33:28.947840 25219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263: Bootstrap starting.
Jun 18 22:33:28 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:28.950980 25219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:28.951932 25219 log.cc:824] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:28.954816 25219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263: No bootstrap required, opened a new log
I20260618 22:33:28.958611 25222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:28.939644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.18.150.190:35785)
0618 22:33:28.940402 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:28.940417 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:28.940455 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:33:28.942032 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:28.942055 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:28.942460 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:28.942830 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:28.944159 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:28.945664 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:28.946913 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:28.947565 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:28.947730 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:28.951652 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:28.951660 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:28.951678 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:28.951713 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:28.954773 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:28.955524 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:28.955530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:28.955536 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:28.955689 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:28.956067 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:28.956070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:28.956072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:28.956580 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:28.956812 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:28.957221 (+ 409us) server_negotiation.cc:299] Negotiation successful
0618 22:33:28.957573 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":330,"threads_started":1}
I20260618 22:33:28.962627 25219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } }
I20260618 22:33:28.962944 25219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:28.963034 25219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9615ea036b408b8b1da7b8b15b4263, State: Initialized, Role: FOLLOWER
I20260618 22:33:28.963568 25219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } }
I20260618 22:33:28.963757 25219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:28.963873 25219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:28.964004 25219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:28.966111 25219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } }
I20260618 22:33:28.966563 25219 leader_election.cc:302] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b9615ea036b408b8b1da7b8b15b4263; no voters:
I20260618 22:33:28.967075 25219 leader_election.cc:288] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:28.967198 25224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:28.967788 25224 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 1 LEADER]: Becoming Leader. State: Replica: 2b9615ea036b408b8b1da7b8b15b4263, State: Running, Role: LEADER
I20260618 22:33:28.968266 25224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } }
I20260618 22:33:28.968952 25219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:28.970927 25225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } }
I20260618 22:33:28.970975 25226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b9615ea036b408b8b1da7b8b15b4263. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } }
I20260618 22:33:28.971269 25225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:28.971305 25226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:28.972019 25232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:28.977134 25232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:28.987365 25232 catalog_manager.cc:1347] Generated new cluster ID: 6965aa89c6e8419c865b29d8b8ae8be6
I20260618 22:33:28.987481 25232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:28.996333 25232 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:28.997526 25232 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:29.004913 25232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263: Generated new TSK 0
I20260618 22:33:29.005640 25232 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab.
Entry for principal oryx/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab.
I20260618 22:33:29.051363 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35785
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab
--principal=oryx/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:29.369897 25245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:29.370268 25245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:29.370434 25245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:29.377678 25245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:29.377825 25245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:29.378026 25245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:33:29.388172 25245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab
--principal=oryx/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:29.389423 25245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:29.390806 25245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:29.403532 25250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:29.403657 25251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:29.403682 25253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:29.404212 25245 server_base.cc:1034] running on GCE node
I20260618 22:33:29.404806 25245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:29.405710 25245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:29.406922 25245 hybrid_clock.cc:648] HybridClock initialized: now 1781822009406877 us; error 31 us; skew 500 ppm
I20260618 22:33:29.407224 25245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:29 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822009, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:29.410148 25245 init.cc:377] Logged in from keytab as oryx/127.18.150.129@KRBTEST.COM (short username oryx)
I20260618 22:33:29.411502 25245 webserver.cc:460] Webserver started at http://127.18.150.129:44433/ using document root <none> and password file <none>
I20260618 22:33:29.412150 25245 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:29.412263 25245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:29.412528 25245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:29.415325 25245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "d3b54272febb41a292c6bf632baaedbc"
format_stamp: "Formatted at 2026-06-18 22:33:29 on dist-test-slave-h7jz"
server_key: "67fff952ec4ce2fcd1f1ca926895ec97"
server_key_iv: "760d9e0c0fca01150fb2ff7a2837d3be"
server_key_version: "encryptionkey@0"
I20260618 22:33:29.416043 25245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "d3b54272febb41a292c6bf632baaedbc"
format_stamp: "Formatted at 2026-06-18 22:33:29 on dist-test-slave-h7jz"
server_key: "67fff952ec4ce2fcd1f1ca926895ec97"
server_key_iv: "760d9e0c0fca01150fb2ff7a2837d3be"
server_key_version: "encryptionkey@0"
I20260618 22:33:29.421062 25245 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:33:29.424304 25260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:29.425513 25245 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:33:29.425647 25245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "d3b54272febb41a292c6bf632baaedbc"
format_stamp: "Formatted at 2026-06-18 22:33:29 on dist-test-slave-h7jz"
server_key: "67fff952ec4ce2fcd1f1ca926895ec97"
server_key_iv: "760d9e0c0fca01150fb2ff7a2837d3be"
server_key_version: "encryptionkey@0"
I20260618 22:33:29.425814 25245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:29.451468 25245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:29.455456 25245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:29.455724 25245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:29.456645 25245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:29.458094 25245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:29.458180 25245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:29.458297 25245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:29.458346 25245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:29.487747 25245 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:43405
I20260618 22:33:29.487783 25372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:43405 every 8 connection(s)
I20260618 22:33:29.488962 25245 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:33:29.489810 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 25245
I20260618 22:33:29.490125 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:33:29.490950 19034 external_mini_cluster.cc:1428] Setting key 4dd5d378c666c8d6fbdbe0b842bfc6bd
Jun 18 22:33:29 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822009, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.150.130@KRBTEST.COM; defaulting to no policy
I20260618 22:33:29.509054 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:29.491721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:51383 (local address 127.18.150.190:35785)
0618 22:33:29.492159 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:29.492164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:29.493169 (+ 1005us) server_negotiation.cc:407] Connection header received
0618 22:33:29.494970 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:29.494975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:29.495046 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:29.495144 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:29.496808 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:29.497847 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:29.499537 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:29.499918 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:29.500023 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:29.503174 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:29.503179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:29.503182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:29.503212 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:29.506114 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:29.506883 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:29.506889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:29.506893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:29.507037 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:29.507453 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:29.507460 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:29.507463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:29.507847 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:29.507999 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:29.508633 (+ 634us) server_negotiation.cc:299] Negotiation successful
0618 22:33:29.508810 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":208,"threads_started":1}
I20260618 22:33:29.510000 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:29.492266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.129:51383)
0618 22:33:29.493015 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:29.493059 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:29.494720 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:29.495395 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:29.495405 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:29.495997 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:29.496646 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:29.496668 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:29.498029 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:29.498034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:29.499400 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:29.499415 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:29.500132 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:29.500140 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:29.500357 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:29.500943 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:29.500964 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:29.502961 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:29.506258 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:29.506267 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:29.506284 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:29.506730 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:29.507157 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:29.507160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:29.507163 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:29.507334 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:29.507958 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:29.507964 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:29.508287 (+ 323us) client_negotiation.cc:769] Sending connection context
0618 22:33:29.508528 (+ 241us) client_negotiation.cc:240] Negotiation successful
0618 22:33:29.508914 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":320,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.130@KRBTEST.COM" created.
I20260618 22:33:29.511744 25373 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35785
I20260618 22:33:29.512037 25373 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:29.512748 25373 heartbeater.cc:511] Master 127.18.150.190:35785 requested a full tablet report, sending...
I20260618 22:33:29.514837 25184 ts_manager.cc:194] Registered new tserver with Master: d3b54272febb41a292c6bf632baaedbc (127.18.150.129:43405)
I20260618 22:33:29.516526 25184 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:51383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab.
Entry for principal oryx/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab.
I20260618 22:33:29.536980 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35785
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab
--principal=oryx/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:29.848810 25379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:29.849092 25379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:29.849181 25379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:29.856345 25379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:29.856454 25379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:29.856662 25379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:33:29.866225 25379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab
--principal=oryx/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:29.867453 25379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:29.868891 25379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:29.881585 25385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:29.881510 25384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:29.881479 25387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:29.882268 25379 server_base.cc:1034] running on GCE node
I20260618 22:33:29.882862 25379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:29.883800 25379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:29.885007 25379 hybrid_clock.cc:648] HybridClock initialized: now 1781822009884954 us; error 41 us; skew 500 ppm
I20260618 22:33:29.885315 25379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:29 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822009, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:29.888355 25379 init.cc:377] Logged in from keytab as oryx/127.18.150.130@KRBTEST.COM (short username oryx)
I20260618 22:33:29.889783 25379 webserver.cc:460] Webserver started at http://127.18.150.130:36235/ using document root <none> and password file <none>
I20260618 22:33:29.890517 25379 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:29.890645 25379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:29.890969 25379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:29.894037 25379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "d7632b2891444502858ae543d072076e"
format_stamp: "Formatted at 2026-06-18 22:33:29 on dist-test-slave-h7jz"
server_key: "6f3efe3dd8a5048b1b702705b3327058"
server_key_iv: "6e821e39b292313c8c523bef67359665"
server_key_version: "encryptionkey@0"
I20260618 22:33:29.894832 25379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "d7632b2891444502858ae543d072076e"
format_stamp: "Formatted at 2026-06-18 22:33:29 on dist-test-slave-h7jz"
server_key: "6f3efe3dd8a5048b1b702705b3327058"
server_key_iv: "6e821e39b292313c8c523bef67359665"
server_key_version: "encryptionkey@0"
I20260618 22:33:29.900043 25379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:33:29.903473 25394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:29.904875 25379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260618 22:33:29.905030 25379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "d7632b2891444502858ae543d072076e"
format_stamp: "Formatted at 2026-06-18 22:33:29 on dist-test-slave-h7jz"
server_key: "6f3efe3dd8a5048b1b702705b3327058"
server_key_iv: "6e821e39b292313c8c523bef67359665"
server_key_version: "encryptionkey@0"
I20260618 22:33:29.905206 25379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:29.920306 25379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:29.923971 25379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:29.924228 25379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:29.925143 25379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:29.926486 25379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:29.926558 25379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:29.926671 25379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:29.926723 25379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:29.957490 25379 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:37773
I20260618 22:33:29.957525 25506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:37773 every 8 connection(s)
I20260618 22:33:29.958770 25379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:33:29.964809 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 25379
I20260618 22:33:29.964970 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:33:29.965376 19034 external_mini_cluster.cc:1428] Setting key 4514d417f28f2ea1315a0d2f99185a72
Jun 18 22:33:29 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822009, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:29.978081 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:29.961367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:33559 (local address 127.18.150.190:35785)
0618 22:33:29.961564 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:29.961569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:29.962756 (+ 1187us) server_negotiation.cc:407] Connection header received
0618 22:33:29.964485 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:29.964490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:29.964571 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:29.964656 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:29.966336 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:29.967157 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:29.968667 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:29.969275 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:29.969389 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:29.972824 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:29.972829 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:29.972832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:29.972862 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:29.975228 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:29.975907 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:29.975911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:29.975913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:29.976037 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:29.976442 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:29.976445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:29.976447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:29.976699 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:29.976834 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:29.977505 (+ 671us) server_negotiation.cc:299] Negotiation successful
0618 22:33:29.977733 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260618 22:33:29.979034 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:29.961859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.130:33559)
0618 22:33:29.962591 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:29.962634 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:29.964214 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:29.964912 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:29.964923 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:29.965489 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:29.966177 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:29.966208 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:29.967293 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:29.967297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:29.968520 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:29.968531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:29.969472 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:29.969480 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:29.969686 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:29.970359 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:29.970381 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:29.972615 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:29.975360 (+ 2745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:29.975369 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:29.975387 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:29.975787 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:29.976136 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:29.976141 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:29.976145 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:29.976332 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:29.976814 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:29.976822 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:29.977157 (+ 335us) client_negotiation.cc:769] Sending connection context
0618 22:33:29.977397 (+ 240us) client_negotiation.cc:240] Negotiation successful
0618 22:33:29.977913 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":276,"threads_started":1}
I20260618 22:33:29.981263 25507 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35785
I20260618 22:33:29.981542 25507 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:29.982309 25507 heartbeater.cc:511] Master 127.18.150.190:35785 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:33:29.983754 25184 ts_manager.cc:194] Registered new tserver with Master: d7632b2891444502858ae543d072076e (127.18.150.130:37773)
I20260618 22:33:29.984542 25184 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:33559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab.
Entry for principal oryx/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab.
I20260618 22:33:30.008644 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35785
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab
--principal=oryx/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:30.318222 25512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:30.318501 25512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:30.318642 25512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:30.325821 25512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:30.325929 25512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:30.326086 25512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:33:30.335830 25512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab
--principal=oryx/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:30.337093 25512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:30.338415 25512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:30.350749 25517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:30.350767 25520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:30.350889 25518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:30.351900 25512 server_base.cc:1034] running on GCE node
I20260618 22:33:30.352495 25512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:30.353427 25512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:30.354604 25512 hybrid_clock.cc:648] HybridClock initialized: now 1781822010354561 us; error 40 us; skew 500 ppm
I20260618 22:33:30.354876 25512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:30 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822010, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:30.362387 25512 init.cc:377] Logged in from keytab as oryx/127.18.150.131@KRBTEST.COM (short username oryx)
I20260618 22:33:30.363833 25512 webserver.cc:460] Webserver started at http://127.18.150.131:37527/ using document root <none> and password file <none>
I20260618 22:33:30.364516 25512 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:30.364621 25512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:30.364894 25512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:30.367775 25512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "f6a71f206db54fe4afde3cc0415e0ba6"
format_stamp: "Formatted at 2026-06-18 22:33:30 on dist-test-slave-h7jz"
server_key: "289b2f5b0ced76174b22abce3e5b7a77"
server_key_iv: "fe8f22086e58fc19520cb8b4f86a8816"
server_key_version: "encryptionkey@0"
I20260618 22:33:30.368482 25512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "f6a71f206db54fe4afde3cc0415e0ba6"
format_stamp: "Formatted at 2026-06-18 22:33:30 on dist-test-slave-h7jz"
server_key: "289b2f5b0ced76174b22abce3e5b7a77"
server_key_iv: "fe8f22086e58fc19520cb8b4f86a8816"
server_key_version: "encryptionkey@0"
I20260618 22:33:30.373440 25512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:33:30.377753 25527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:30.379542 25512 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:33:30.379735 25512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "f6a71f206db54fe4afde3cc0415e0ba6"
format_stamp: "Formatted at 2026-06-18 22:33:30 on dist-test-slave-h7jz"
server_key: "289b2f5b0ced76174b22abce3e5b7a77"
server_key_iv: "fe8f22086e58fc19520cb8b4f86a8816"
server_key_version: "encryptionkey@0"
I20260618 22:33:30.379993 25512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:30.419431 25512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:30.423782 25512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:30.424074 25512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:30.425079 25512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:30.426517 25512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:30.426596 25512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:30.426712 25512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:30.426761 25512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:30.458293 25512 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:38539
I20260618 22:33:30.458303 25639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:38539 every 8 connection(s)
I20260618 22:33:30.459554 25512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:33:30.467479 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 25512
I20260618 22:33:30.467656 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:33:30.468183 19034 external_mini_cluster.cc:1428] Setting key 02b1057126c75c3d610881e41471505d
Jun 18 22:33:30 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822010, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:30.481897 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:30.462152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:43729 (local address 127.18.150.190:35785)
0618 22:33:30.462322 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:30.462327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:30.463813 (+ 1486us) server_negotiation.cc:407] Connection header received
0618 22:33:30.465594 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:30.465601 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:30.465695 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:30.465784 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:30.467512 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:30.468393 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:30.470554 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:30.471287 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:30.471412 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:30.475110 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:30.475117 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:30.475122 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:30.475164 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:30.478600 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:30.479358 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:30.479363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:30.479367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:30.479510 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:30.479979 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:30.479984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:30.479988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:30.480360 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:30.480501 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:30.481310 (+ 809us) server_negotiation.cc:299] Negotiation successful
0618 22:33:30.481543 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:33:30.483245 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:30.462878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.131:43729)
0618 22:33:30.463620 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:30.463658 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:30.465300 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:30.466065 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:30.466074 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:30.466579 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:30.467318 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:30.467339 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:30.468538 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:30.468542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:30.470388 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:30.470410 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:30.471490 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:30.471495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:30.471703 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:30.472439 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:30.472460 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:30.474865 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:30.478786 (+ 3921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:30.478794 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:30.478811 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:30.479241 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:30.479649 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:30.479655 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:30.479658 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:30.479878 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:30.480488 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:30.480494 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:30.480869 (+ 375us) client_negotiation.cc:769] Sending connection context
0618 22:33:30.481152 (+ 283us) client_negotiation.cc:240] Negotiation successful
0618 22:33:30.481650 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":289,"threads_started":1}
I20260618 22:33:30.484879 25640 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35785
I20260618 22:33:30.485198 25640 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:30.485885 25640 heartbeater.cc:511] Master 127.18.150.190:35785 requested a full tablet report, sending...
I20260618 22:33:30.487337 25184 ts_manager.cc:194] Registered new tserver with Master: f6a71f206db54fe4afde3cc0415e0ba6 (127.18.150.131:38539)
I20260618 22:33:30.488139 25184 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:43729
I20260618 22:33:30.496111 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:33:30.520134 25373 heartbeater.cc:503] Master 127.18.150.190:35785 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.18.150.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.189.keytab.
Entry for principal oryx/127.18.150.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.189.keytab.
I20260618 22:33:30.554055 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.189:37051
--webserver_interface=127.18.150.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.189.keytab
--principal=oryx/127.18.150.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:35785,127.18.150.189:37051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:30.879253 25645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:30.879528 25645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:30.879649 25645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:30.886986 25645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:30.887099 25645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:30.887171 25645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:30.887246 25645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:30.887298 25645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:30.897320 25645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:35785,127.18.150.189:37051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.189.keytab
--principal=oryx/127.18.150.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.189:37051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.150.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
W20260618 22:33:30.897683 25645 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.150.190:35785,127.18.150.189:37051'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260618 22:33:30.898787 25645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:30.900120 25645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:30.911409 25653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:30.911459 25651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:30.911523 25650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:30.912664 25645 server_base.cc:1034] running on GCE node
I20260618 22:33:30.913441 25645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:30.914837 25645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:30.916051 25645 hybrid_clock.cc:648] HybridClock initialized: now 1781822010915993 us; error 41 us; skew 500 ppm
I20260618 22:33:30.916369 25645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:30 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822010, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:30.919423 25645 init.cc:377] Logged in from keytab as oryx/127.18.150.189@KRBTEST.COM (short username oryx)
I20260618 22:33:30.921049 25645 webserver.cc:460] Webserver started at http://127.18.150.189:42065/ using document root <none> and password file <none>
I20260618 22:33:30.921715 25645 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:30.921798 25645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:30.922099 25645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:30.924973 25645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data/instance:
uuid: "863af7c92037482dbd2d470608c0a984"
format_stamp: "Formatted at 2026-06-18 22:33:30 on dist-test-slave-h7jz"
server_key: "e0c79d29ebec2c2ee1397ca7724a82a6"
server_key_iv: "bcc7ee916ded4c2f55bdcee5076d00e2"
server_key_version: "encryptionkey@0"
I20260618 22:33:30.925704 25645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/wal/instance:
uuid: "863af7c92037482dbd2d470608c0a984"
format_stamp: "Formatted at 2026-06-18 22:33:30 on dist-test-slave-h7jz"
server_key: "e0c79d29ebec2c2ee1397ca7724a82a6"
server_key_iv: "bcc7ee916ded4c2f55bdcee5076d00e2"
server_key_version: "encryptionkey@0"
I20260618 22:33:30.930840 25645 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260618 22:33:30.934207 25660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:30.935642 25645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:33:30.935796 25645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/wal
uuid: "863af7c92037482dbd2d470608c0a984"
format_stamp: "Formatted at 2026-06-18 22:33:30 on dist-test-slave-h7jz"
server_key: "e0c79d29ebec2c2ee1397ca7724a82a6"
server_key_iv: "bcc7ee916ded4c2f55bdcee5076d00e2"
server_key_version: "encryptionkey@0"
I20260618 22:33:30.935978 25645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:30.948755 25645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:30.952919 25645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:30.953159 25645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:30.969780 25645 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.189:37051
I20260618 22:33:30.969764 25711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.189:37051 every 8 connection(s)
I20260618 22:33:30.971120 25645 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
I20260618 22:33:30.974673 25712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:30.975906 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 25645
I20260618 22:33:30.983158 25712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:33:30.987116 25507 heartbeater.cc:503] Master 127.18.150.190:35785 was elected leader, sending a full tablet report...
Jun 18 22:33:30 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.150.189@KRBTEST.COM
Jun 18 22:33:30 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822010, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.189@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:30.998556 25715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:30.978232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53264 (local address 127.18.150.189:37051)
0618 22:33:30.978955 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:30.978967 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:30.979010 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:33:30.980660 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:30.980684 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:30.981149 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:30.981522 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:30.982961 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:30.984452 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:30.985783 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:30.986668 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:30.986880 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:30.991153 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:30.991160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:30.991170 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:30.991210 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:30.994393 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:30.995088 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:30.995095 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:30.995101 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:30.995242 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:30.995648 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:30.995652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:30.995655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:30.996220 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:30.996435 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:30.996936 (+ 501us) server_negotiation.cc:299] Negotiation successful
0618 22:33:30.997301 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":304,"threads_started":1}
I20260618 22:33:31.000815 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:30.985624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60600 (local address 127.18.150.190:35785)
0618 22:33:30.986121 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:30.986126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:30.986715 (+ 589us) server_negotiation.cc:407] Connection header received
0618 22:33:30.987088 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:30.987093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:30.987158 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:30.987263 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:30.988703 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:30.989415 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:30.990916 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:30.991440 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:30.991535 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:30.995025 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:30.995032 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:30.995036 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:30.995082 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:30.997931 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:30.998757 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:30.998763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:30.998767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:30.998992 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:30.999342 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:30.999345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:30.999347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:30.999691 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:30.999816 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.000257 (+ 441us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.000454 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":249,"threads_started":1}
I20260618 22:33:31.000870 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:30.985612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:60600)
0618 22:33:30.986572 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:30.986621 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:30.986764 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:30.987459 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:30.987466 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:30.988225 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:30.988540 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:30.988552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:30.989547 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:30.989552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:30.990780 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:30.990791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:30.991619 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:30.991625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:30.991791 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:30.992288 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:30.992325 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:30.994835 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:30.998083 (+ 3248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:30.998091 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:30.998098 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:30.998614 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:30.999091 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:30.999095 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:30.999099 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:30.999253 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:30.999813 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:30.999820 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:30.999971 (+ 151us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.000147 (+ 176us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.000474 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":529,"threads_started":1}
I20260618 22:33:31.002427 25712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:33:31 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822010, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.189@KRBTEST.COM for oryx/127.18.150.189@KRBTEST.COM
I20260618 22:33:31.013973 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.003021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:37051 (local address 127.0.0.1:53266)
0618 22:33:31.003186 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.003203 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.003302 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.003545 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.003549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.003878 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.003994 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.004002 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.004913 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.004917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.005744 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.005763 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.006450 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.006455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.006599 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.007068 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.007083 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.009150 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.011766 (+ 2616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.011770 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.011772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.012142 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.012460 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.012464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.012466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.012595 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.013172 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.013175 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.013291 (+ 116us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.013400 (+ 109us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.013683 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260618 22:33:31.014153 25715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.003045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.18.150.189:37051)
0618 22:33:31.003185 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.003191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.003267 (+ 76us) server_negotiation.cc:407] Connection header received
0618 22:33:31.003397 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.003402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.003451 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.003519 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.004109 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.004790 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.005869 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.006326 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.006429 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.009307 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.009311 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.009314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.009340 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.011639 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.012234 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.012238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.012240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.012349 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.012722 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.012726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.012728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.013017 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.013155 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.013495 (+ 340us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.013795 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":41}
I20260618 22:33:31.018568 25712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: Bootstrap starting.
I20260618 22:33:31.021565 25712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:31.022609 25712 log.cc:824] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:31.025141 25712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: No bootstrap required, opened a new log
I20260618 22:33:31.031895 25712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } }
I20260618 22:33:31.032243 25712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:31.032341 25712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863af7c92037482dbd2d470608c0a984, State: Initialized, Role: FOLLOWER
I20260618 22:33:31.032929 25712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } }
I20260618 22:33:31.033775 25719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } } }
I20260618 22:33:31.034097 25719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:33:31.034793 25712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: configured and running, proceeding with master startup.
W20260618 22:33:31.043215 25730 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:33:31.043368 25730 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:33:31.045246 25645 master_runner.cc:428] Detected that this master 863af7c92037482dbd2d470608c0a984 is joining an existing cluster
I20260618 22:33:31.045334 25645 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.150.189:37051
I20260618 22:33:31.064131 25715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.048902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53274 (local address 127.18.150.189:37051)
0618 22:33:31.049102 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.049108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.050017 (+ 909us) server_negotiation.cc:407] Connection header received
0618 22:33:31.050235 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.050241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.050303 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.050378 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.051563 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.052632 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.054479 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.055225 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.055348 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.057660 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.057666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.057683 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.057715 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.061384 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.062186 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.062195 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.062199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.062301 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.062634 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.062640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.062643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.063011 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.063156 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.063505 (+ 349us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.063810 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":127,"server-negotiator.queue_time_us":54}
I20260618 22:33:31.064162 25736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.049090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:37051 (local address 127.0.0.1:53274)
0618 22:33:31.049888 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.049909 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.050121 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.050511 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.050515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.051135 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.051435 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.051443 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.052832 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.052837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.054347 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.054359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.055414 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.055422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.055573 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.056266 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.056286 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.057498 (+ 1212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.061529 (+ 4031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.061534 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.061538 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.062058 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.062399 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.062404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.062407 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.062532 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.063108 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.063116 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.063251 (+ 135us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.063372 (+ 121us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.063679 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":3,"thread_start_us":452,"threads_started":1}
I20260618 22:33:31.064877 25737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.049463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:60616)
0618 22:33:31.050252 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.050270 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.050388 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.050716 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.050721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.051415 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.051807 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.051819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.053237 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.053242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.055603 (+ 2361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.055614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.056758 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.056765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.056953 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.057502 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.057525 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.058635 (+ 1110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.062142 (+ 3507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.062147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.062152 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.062590 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.062945 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.062950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.062954 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.063107 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.063792 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.063797 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.063976 (+ 179us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.064104 (+ 128us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.064428 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":4,"thread_start_us":497,"threads_started":1}
I20260618 22:33:31.066325 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.048918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.18.150.190:35785)
0618 22:33:31.049081 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.049087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.050358 (+ 1271us) server_negotiation.cc:407] Connection header received
0618 22:33:31.050488 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.050493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.050576 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.050680 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.051979 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.053033 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.055773 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.056383 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.056477 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.058848 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.058854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.058859 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.058897 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.062026 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.062702 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.062708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.062711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.062837 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.063241 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.063246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.063249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.063583 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.063726 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.065789 (+ 2063us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.066036 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for oryx/127.18.150.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.188@KRBTEST.COM" created.
Jun 18 22:33:31 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822008, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.190@KRBTEST.COM for oryx/127.18.150.189@KRBTEST.COM
I20260618 22:33:31.090854 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.074626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:37051 (local address 127.0.0.1:53284)
0618 22:33:31.075321 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.075362 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.075533 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.076141 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.076149 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.076809 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.077018 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.077026 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.078415 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.078419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.079643 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.079657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.080658 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.080664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.080848 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.081558 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.081577 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.084231 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.088085 (+ 3854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.088098 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.088106 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.088583 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.088963 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.088968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.088971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.089119 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.089761 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.089769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.089987 (+ 218us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.090178 (+ 191us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.090483 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":364,"threads_started":1}
I20260618 22:33:31.090900 25715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.074972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53284 (local address 127.18.150.189:37051)
0618 22:33:31.075761 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.075767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.075789 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:33:31.075874 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.075879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.075977 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.076079 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.077160 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.078290 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.079786 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.080514 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.080637 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.084414 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.084421 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.084426 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.084463 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.087950 (+ 3487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.088700 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.088706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.088710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.088858 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.089225 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.089230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.089233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.089620 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.089772 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.090302 (+ 530us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.090528 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671}
I20260618 22:33:31.093272 25184 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.150.189:37051: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "863af7c92037482dbd2d470608c0a984"
member_type: NON_VOTER
last_known_addr {
host: "127.18.150.189"
port: 37051
}
attrs {
promote: true
}
}
dest_uuid: "2b9615ea036b408b8b1da7b8b15b4263"
cas_config_opid_index: -1
I20260618 22:33:31.094466 25184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } }
I20260618 22:33:31.098608 25742 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 863af7c92037482dbd2d470608c0a984 (127.18.150.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } } }
I20260618 22:33:31.099994 25687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:31.103463 25742 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.150.189:37051
I20260618 22:33:31.103642 25687 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 FOLLOWER]: Refusing update from remote peer 2b9615ea036b408b8b1da7b8b15b4263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260618 22:33:31.104229 25645 master.cc:537] Master@127.18.150.189:37051 shutting down...
I20260618 22:33:31.104046 25743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } } }
I20260618 22:33:31.105190 25743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:31.105158 25742 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.188.keytab.
Entry for principal oryx/127.18.150.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.188.keytab.
I20260618 22:33:31.107623 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.188:34705
--webserver_interface=127.18.150.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.188.keytab
--principal=oryx/127.18.150.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:35785,127.18.150.189:37051,127.18.150.188:34705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:31.109283 25171 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260618 22:33:31.114337 25171 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 -> Peer 863af7c92037482dbd2d470608c0a984 (127.18.150.189:37051): Couldn't send request to peer 863af7c92037482dbd2d470608c0a984. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260618 22:33:31.118973 25645 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:31.119356 25645 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:31.119526 25645 tablet_replica.cc:330] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: stopping tablet replica
I20260618 22:33:31.135435 25645 master.cc:552] Master@127.18.150.189:37051 shutdown complete.
I20260618 22:33:31.135520 25645 master_runner.cc:305] Clearing existing system tablet
I20260618 22:33:31.135908 25645 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:31.135994 25645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:31.138417 25645 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260618 22:33:31.139732 25748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:31.140008 25645 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:31.140259 25645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260618 22:33:31.140337 25645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/wal
uuid: "863af7c92037482dbd2d470608c0a984"
format_stamp: "Formatted at 2026-06-18 22:33:30 on dist-test-slave-h7jz"
server_key: "e0c79d29ebec2c2ee1397ca7724a82a6"
server_key_iv: "bcc7ee916ded4c2f55bdcee5076d00e2"
server_key_version: "encryptionkey@0"
I20260618 22:33:31.143602 25645 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:31.148234 25645 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260618 22:33:31.148386 25645 log.cc:1196] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260618 22:33:31.148737 25645 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: Deleting consensus metadata
I20260618 22:33:31.148947 25645 master_runner.cc:315] Copying system tablet from 127.18.150.190:35785
I20260618 22:33:31.151032 25645 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: tablet copy: Beginning tablet copy session from remote peer at address 127.18.150.190:35785
I20260618 22:33:31.162235 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.151701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.18.150.190:35785)
0618 22:33:31.151893 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.151901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.152321 (+ 420us) server_negotiation.cc:407] Connection header received
0618 22:33:31.152500 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.152505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.152570 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.152651 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.153420 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.154179 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.155415 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.155835 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.155939 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.157593 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.157598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.157602 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.157629 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.159944 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.160486 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.160491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.160495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.160603 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.160929 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.160934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.160938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.161180 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.161327 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.161676 (+ 349us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.161860 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:33:31.162308 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.151667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:60628)
0618 22:33:31.152198 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.152222 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.152364 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.152685 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.152689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.153096 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.153291 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.153300 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.154312 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.154317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.155291 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.155303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.155963 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.155969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.156152 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.156712 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.156728 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.157421 (+ 693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.160054 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.160059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.160064 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.160388 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.160696 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.160701 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.160705 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.160837 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.161284 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.161289 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.161439 (+ 150us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.161557 (+ 118us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.161847 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":331,"threads_started":1}
I20260618 22:33:31.163290 25204 tablet_copy_service.cc:140] P 2b9615ea036b408b8b1da7b8b15b4263: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 863af7c92037482dbd2d470608c0a984 ({username='oryx', principal='oryx/127.18.150.189@KRBTEST.COM'} at 127.0.0.1:60628)
I20260618 22:33:31.163506 25204 tablet_copy_service.cc:161] P 2b9615ea036b408b8b1da7b8b15b4263: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 863af7c92037482dbd2d470608c0a984 at {username='oryx', principal='oryx/127.18.150.189@KRBTEST.COM'} at 127.0.0.1:60628: session id = 863af7c92037482dbd2d470608c0a984-00000000000000000000000000000000
I20260618 22:33:31.166543 25204 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263: Tablet Copy: opened 0 blocks and 1 log segments
I20260618 22:33:31.168231 25645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260618 22:33:31.172634 25645 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: tablet copy: Starting download of 0 data blocks...
I20260618 22:33:31.172886 25645 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: tablet copy: Starting download of 1 WAL segments...
I20260618 22:33:31.175050 25645 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260618 22:33:31.178083 25204 tablet_copy_service.cc:342] P 2b9615ea036b408b8b1da7b8b15b4263: Request end of tablet copy session 863af7c92037482dbd2d470608c0a984-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.150.189@KRBTEST.COM'} at 127.0.0.1:60628
I20260618 22:33:31.178246 25204 tablet_copy_service.cc:433] P 2b9615ea036b408b8b1da7b8b15b4263: ending tablet copy session 863af7c92037482dbd2d470608c0a984-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 863af7c92037482dbd2d470608c0a984
I20260618 22:33:31.187929 25645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:31.190326 25755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:31.196550 25756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:31.201330 25758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:31.203526 25645 server_base.cc:1034] running on GCE node
I20260618 22:33:31.203821 25645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:31.204398 25645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:31.205585 25645 hybrid_clock.cc:648] HybridClock initialized: now 1781822011205530 us; error 33 us; skew 500 ppm
I20260618 22:33:31.205785 25645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:31 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822011, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:31.207443 25645 init.cc:377] Logged in from keytab as oryx/127.18.150.189@KRBTEST.COM (short username oryx)
I20260618 22:33:31.208613 25645 webserver.cc:460] Webserver started at http://127.18.150.189:33769/ using document root <none> and password file <none>
I20260618 22:33:31.208909 25645 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:31.208979 25645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:31.211293 25645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260618 22:33:31.212767 25765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:31.213095 25645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260618 22:33:31.213217 25645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/wal
uuid: "863af7c92037482dbd2d470608c0a984"
format_stamp: "Formatted at 2026-06-18 22:33:30 on dist-test-slave-h7jz"
server_key: "e0c79d29ebec2c2ee1397ca7724a82a6"
server_key_iv: "bcc7ee916ded4c2f55bdcee5076d00e2"
server_key_version: "encryptionkey@0"
I20260618 22:33:31.213354 25645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:31.223769 25645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:31.226902 25645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:31.248430 25645 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.189:37051
I20260618 22:33:31.248430 25816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.189:37051 every 8 connection(s)
I20260618 22:33:31.249294 25645 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
I20260618 22:33:31.254788 25817 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:33:31.257889 25817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: Bootstrap starting.
I20260618 22:33:31.270133 25817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:33:31.270588 25817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: Bootstrap complete.
I20260618 22:33:31.271528 25817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } }
I20260618 22:33:31.271739 25817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 863af7c92037482dbd2d470608c0a984, State: Initialized, Role: LEARNER
I20260618 22:33:31.271960 25817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } }
I20260618 22:33:31.272687 25820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } } }
I20260618 22:33:31.273020 25817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:31.273167 25820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: This master's current role is: LEARNER
I20260618 22:33:31.286489 25831 catalog_manager.cc:1259] Loaded cluster ID: 6965aa89c6e8419c865b29d8b8ae8be6
I20260618 22:33:31.286602 25831 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: loading cluster ID for follower catalog manager: success
I20260618 22:33:31.289249 25831 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: acquiring CA information for follower catalog manager: success
I20260618 22:33:31.291383 25831 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:33:31.490545 25640 heartbeater.cc:503] Master 127.18.150.190:35785 was elected leader, sending a full tablet report...
W20260618 22:33:31.525609 25746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:31.525885 25746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:31.526007 25746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:31.533334 25746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:31.533439 25746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:31.533483 25746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:31.533537 25746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:31.533577 25746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:31.545316 25746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:35785,127.18.150.189:37051,127.18.150.188:34705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.188.keytab
--principal=oryx/127.18.150.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.188:34705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.150.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:31.546299 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.539464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:37051 (local address 127.0.0.1:53288)
0618 22:33:31.539655 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.539676 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.539806 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.540587 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.540591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.540617 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:31.540757 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.540764 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.542687 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.542691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.543995 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.544008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.545591 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.545599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.545841 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.545888 (+ 47us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.546003 (+ 115us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.546107 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260618 22:33:31.546455 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.539655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53288 (local address 127.18.150.189:37051)
0618 22:33:31.540227 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.540233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.540252 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:33:31.540322 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.540327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.540413 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.540539 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:31.540883 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.542513 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.544151 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.545386 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.545571 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.545774 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.546029 (+ 255us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.546158 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":305,"threads_started":1}
I20260618 22:33:31.547006 25746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:31.547607 25792 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260618 22:33:31.548223 25820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b9615ea036b408b8b1da7b8b15b4263. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } } }
I20260618 22:33:31.548455 25820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: This master's current role is: LEARNER
I20260618 22:33:31.549010 25746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:31.560338 25837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:31.560343 25840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:31.560379 25838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:31.561024 25746 server_base.cc:1034] running on GCE node
I20260618 22:33:31.561793 25746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:31.563289 25746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:31.564483 25746 hybrid_clock.cc:648] HybridClock initialized: now 1781822011564433 us; error 48 us; skew 500 ppm
I20260618 22:33:31.564808 25746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:31 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822011, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:31.567792 25746 init.cc:377] Logged in from keytab as oryx/127.18.150.188@KRBTEST.COM (short username oryx)
I20260618 22:33:31.569258 25746 webserver.cc:460] Webserver started at http://127.18.150.188:38671/ using document root <none> and password file <none>
I20260618 22:33:31.569923 25746 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:31.570070 25746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:31.570350 25746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:31.573271 25746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data/instance:
uuid: "0857f29096284b568f1731395c320ab1"
format_stamp: "Formatted at 2026-06-18 22:33:31 on dist-test-slave-h7jz"
server_key: "f0e91c9c81736a9cfea9d2dcd894e478"
server_key_iv: "1bf0affdf4c3143ea09535f37ccc1175"
server_key_version: "encryptionkey@0"
I20260618 22:33:31.574077 25746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/wal/instance:
uuid: "0857f29096284b568f1731395c320ab1"
format_stamp: "Formatted at 2026-06-18 22:33:31 on dist-test-slave-h7jz"
server_key: "f0e91c9c81736a9cfea9d2dcd894e478"
server_key_iv: "1bf0affdf4c3143ea09535f37ccc1175"
server_key_version: "encryptionkey@0"
I20260618 22:33:31.578935 25746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260618 22:33:31.582360 25847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:31.583714 25746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:33:31.583866 25746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/wal
uuid: "0857f29096284b568f1731395c320ab1"
format_stamp: "Formatted at 2026-06-18 22:33:31 on dist-test-slave-h7jz"
server_key: "f0e91c9c81736a9cfea9d2dcd894e478"
server_key_iv: "1bf0affdf4c3143ea09535f37ccc1175"
server_key_version: "encryptionkey@0"
I20260618 22:33:31.584034 25746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:31.599805 25746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:31.604024 25746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:31.604254 25746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:31.621529 25746 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.188:34705
I20260618 22:33:31.621563 25898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.188:34705 every 8 connection(s)
I20260618 22:33:31.622908 25746 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
I20260618 22:33:31.623453 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 25746
I20260618 22:33:31.627234 25899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:31.634727 25899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:33:31 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.150.188@KRBTEST.COM
Jun 18 22:33:31 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822011, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.188@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:31.645855 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.625619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49646 (local address 127.18.150.188:34705)
0618 22:33:31.626361 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.626373 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.626413 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:33:31.627938 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.627959 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.628341 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.628679 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.630145 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.631457 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.633180 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.633835 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.634003 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.639104 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.639112 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.639122 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.639159 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.641803 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.642459 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.642465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.642471 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.642616 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.642983 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.642987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.642989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.643558 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.643870 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.644261 (+ 391us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.644625 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":335,"threads_started":1}
I20260618 22:33:31.650509 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.637838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.18.150.190:35785)
0618 22:33:31.638021 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.638027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.638607 (+ 580us) server_negotiation.cc:407] Connection header received
0618 22:33:31.638777 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.638782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.638843 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.638932 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.640041 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.640721 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.642113 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.642481 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.642584 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.645752 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.645757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.645760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.645790 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.648303 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.648937 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.648941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.648943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.649062 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.649378 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.649381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.649384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.649640 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.649747 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.650134 (+ 387us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.650287 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:33:31.650539 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.637777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:60638)
0618 22:33:31.638477 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.638511 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.638656 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.639001 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.639008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.639663 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.639899 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.639912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.640852 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.640858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.641990 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.642005 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.642648 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.642654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.642819 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.643340 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.643355 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.645553 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.648417 (+ 2864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.648425 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.648431 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.648839 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.649154 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.649157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.649160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.649295 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.649728 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.649732 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.649869 (+ 137us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.650038 (+ 169us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.650261 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":380,"threads_started":1}
I20260618 22:33:31.651696 25899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:33:31 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822011, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.188@KRBTEST.COM for oryx/127.18.150.189@KRBTEST.COM
I20260618 22:33:31.662703 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.652413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.18.150.189:37051)
0618 22:33:31.652557 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.652561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.652603 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:33:31.652712 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.652716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.652779 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.652849 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.653356 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.654082 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.655183 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.655589 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.655677 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.658444 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.658449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.658452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.658481 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.660723 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.661274 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.661277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.661279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.661388 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.661677 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.661680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.661682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.661933 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.662064 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.662322 (+ 258us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.662480 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:33:31.662686 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.652372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:37051 (local address 127.0.0.1:53290)
0618 22:33:31.652532 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.652549 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.652636 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.652859 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.652862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.653141 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.653261 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.653268 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.654169 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.654172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.655056 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.655065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.655779 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.655784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.655913 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.656411 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.656427 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.658276 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.660844 (+ 2568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.660848 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.660850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.661187 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.661473 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.661477 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.661479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.661606 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.662035 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.662038 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.662149 (+ 111us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.662255 (+ 106us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.662459 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260618 22:33:31.663681 25899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:33:31 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822011, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.188@KRBTEST.COM for oryx/127.18.150.188@KRBTEST.COM
I20260618 22:33:31.674765 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.664363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:34705 (local address 127.0.0.1:49648)
0618 22:33:31.664533 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.664547 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.664621 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.665009 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.665013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.665331 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.665466 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.665474 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.666453 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.666457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.667287 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.667297 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.667894 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.667897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.668037 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.668536 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.668552 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.670335 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.672704 (+ 2369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.672708 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.672711 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.673100 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.673377 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.673380 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.673382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.673511 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.673974 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.673978 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.674138 (+ 160us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.674263 (+ 125us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.674442 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260618 22:33:31.674816 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.664363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.18.150.188:34705)
0618 22:33:31.664753 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.664757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.664767 (+ 10us) server_negotiation.cc:407] Connection header received
0618 22:33:31.664795 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.664798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.664841 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.664900 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.665595 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.666347 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.667391 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.667776 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.667860 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.670486 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.670491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.670494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.670519 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.672590 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.673184 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.673188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.673191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.673298 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.673587 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.673590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.673592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.673857 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.674009 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.674333 (+ 324us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.674552 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":311}
I20260618 22:33:31.679337 25899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: Bootstrap starting.
I20260618 22:33:31.682433 25899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:31.683423 25899 log.cc:824] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:31.685987 25899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: No bootstrap required, opened a new log
I20260618 22:33:31.692816 25899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } }
I20260618 22:33:31.693115 25899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:31.693222 25899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0857f29096284b568f1731395c320ab1, State: Initialized, Role: FOLLOWER
I20260618 22:33:31.693761 25899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } }
I20260618 22:33:31.694541 25905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } } }
I20260618 22:33:31.694813 25905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:33:31.695461 25899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:31.700723 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 25245
W20260618 22:33:31.704087 25916 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:33:31.704216 25916 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:33:31.707525 25746 master_runner.cc:428] Detected that this master 0857f29096284b568f1731395c320ab1 is joining an existing cluster
I20260618 22:33:31.707628 25746 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.150.188:34705
Jun 18 22:33:31 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.150.129@KRBTEST.COM
I20260618 22:33:31.738724 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.702373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43048 (local address 127.18.150.129:43405)
0618 22:33:31.706569 (+ 4196us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.706578 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.706606 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:33:31.706668 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.706678 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.706936 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.707102 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.708768 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.710374 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.712339 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.713794 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.719626 (+ 5832us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.719937 (+ 311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.719951 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.719969 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.720008 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.723882 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.733769 (+ 9887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.733780 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.733784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.734001 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.734517 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.734521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.734525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.734995 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.737824 (+ 2829us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.738019 (+ 195us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.738321 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4027,"thread_start_us":3706,"threads_started":1}
I20260618 22:33:31.742103 25368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:31.757253 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.712823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:60646)
0618 22:33:31.726488 (+ 13665us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.726509 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.734076 (+ 7567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.734664 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.734669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.735293 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.735679 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.735704 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.737125 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.737131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.738735 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.738750 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.739927 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.739935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.740094 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.741781 (+ 1687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.741803 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.743122 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.746819 (+ 3697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.746824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.746828 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.754555 (+ 7727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.755060 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.755065 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.755068 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.755266 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.755958 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.755964 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.756222 (+ 258us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.756402 (+ 180us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.756718 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13469,"thread_start_us":43132,"threads_started":1}
I20260618 22:33:31.757983 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.713013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.18.150.190:35785)
0618 22:33:31.713209 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.713215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.726605 (+ 13390us) server_negotiation.cc:407] Connection header received
0618 22:33:31.734261 (+ 7656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.734267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.734352 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.734427 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.735863 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.736948 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.738901 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.739529 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.739629 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.743327 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.743332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.743337 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.743379 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.746654 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.754746 (+ 8092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.754752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.754755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.754934 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.755408 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.755413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.755417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.755796 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.755955 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.757373 (+ 1418us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.757628 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:33:31.772853 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.712828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.18.150.188:34705)
0618 22:33:31.713343 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.713348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.754625 (+ 41277us) server_negotiation.cc:407] Connection header received
0618 22:33:31.754850 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.754856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.754928 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.754999 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.756379 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.757503 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.760764 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.761486 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.761582 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.766170 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.766178 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.766183 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.766215 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.769815 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.770681 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.770686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.770690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.770838 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.771176 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.771181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.771185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.771572 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.771715 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.772207 (+ 492us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.772455 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Jun 18 22:33:31 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822008, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.190@KRBTEST.COM for oryx/127.18.150.188@KRBTEST.COM
I20260618 22:33:31.785194 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.713279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53292 (local address 127.18.150.189:37051)
0618 22:33:31.713426 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.713432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.756389 (+ 42957us) server_negotiation.cc:407] Connection header received
0618 22:33:31.756533 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.756538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.756611 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.756693 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.759525 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.760572 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.762677 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.763322 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.763398 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.770215 (+ 6817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.770222 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.770227 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.770268 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.773536 (+ 3268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.782764 (+ 9228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.782770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.782774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.782949 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.783441 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.783447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.783450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.783809 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.783966 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.784596 (+ 630us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.784847 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:33:31.786278 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.719168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:37051 (local address 127.0.0.1:53292)
0618 22:33:31.756293 (+ 37125us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.756318 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.756420 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.756862 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.756867 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.758884 (+ 2017us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.759378 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.759392 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.760707 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.760712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.762515 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.762528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.763680 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.763686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.763906 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.764555 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.764582 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.765777 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.782040 (+ 16263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.782045 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.782050 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.782603 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.783101 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.783105 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.783110 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.783311 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.784065 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.784070 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.784313 (+ 243us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.785456 (+ 1143us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.785782 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36984,"thread_start_us":6929,"threads_started":1}
I20260618 22:33:31.787824 25924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.713189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:34705 (local address 127.0.0.1:49654)
0618 22:33:31.754501 (+ 41312us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.754524 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.754729 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.755208 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.755212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.755865 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.756237 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.756250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.757700 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.757707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.760631 (+ 2924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.760644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.761898 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.761905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.762104 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.762726 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.762748 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.766003 (+ 3255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.770001 (+ 3998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.770007 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.770011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.770561 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.770955 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.770960 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.770964 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.771087 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.771776 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.771780 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.771956 (+ 176us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.787027 (+ 15071us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.787354 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41145,"mutex_wait_us":429,"thread_start_us":40934,"threads_started":1}
I20260618 22:33:31.791188 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.763185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:34705 (local address 127.0.0.1:49666)
0618 22:33:31.763571 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.763590 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.763702 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.773677 (+ 9975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.773683 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.774174 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.774420 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.774434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.776162 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.776169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.777643 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.777660 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.778811 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.778819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.779010 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.779771 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.779797 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.782777 (+ 2980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.788450 (+ 5673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.788455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.788459 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.788931 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.789615 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.789618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.789621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.789748 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.790286 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.790291 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.790426 (+ 135us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.790538 (+ 112us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.790828 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262}
I20260618 22:33:31.791178 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.763543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49666 (local address 127.18.150.188:34705)
0618 22:33:31.773193 (+ 9650us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.773199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.773218 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:33:31.773343 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.773347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.773472 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.774578 (+ 1106us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.774809 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.775976 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.777849 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.778619 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.782964 (+ 4345us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.783166 (+ 202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.783173 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.783178 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.783208 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.786925 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.789391 (+ 2466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.789394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.789397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.789500 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.789858 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.789863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.789867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.790169 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.790292 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.790661 (+ 369us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.790854 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9540,"thread_start_us":24047,"threads_started":1}
I20260618 22:33:31.792667 25184 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.150.188:34705: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0857f29096284b568f1731395c320ab1"
member_type: NON_VOTER
last_known_addr {
host: "127.18.150.188"
port: 34705
}
attrs {
promote: true
}
}
dest_uuid: "2b9615ea036b408b8b1da7b8b15b4263"
cas_config_opid_index: 5
I20260618 22:33:31.793475 25184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } }
I20260618 22:33:31.794987 25742 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0857f29096284b568f1731395c320ab1 (127.18.150.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:31.795696 25792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 LEARNER]: Refusing update from remote peer 2b9615ea036b408b8b1da7b8b15b4263: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260618 22:33:31.796996 25928 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260618 22:33:31.798767 25742 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.150.188:34705
I20260618 22:33:31.799046 25742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:31.799280 25742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:31.799412 25746 master.cc:537] Master@127.18.150.188:34705 shutting down...
I20260618 22:33:31.800174 25874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:31.801029 25792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0857f29096284b568f1731395c320ab1 (127.18.150.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:31.803845 25820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: NON_VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: true } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:31.804106 25820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: This master's current role is: LEARNER
I20260618 22:33:31.804500 25928 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263: attempting to promote NON_VOTER 863af7c92037482dbd2d470608c0a984 to VOTER
I20260618 22:33:31.805230 25928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } }
I20260618 22:33:31.807503 25792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 LEARNER]: Refusing update from remote peer 2b9615ea036b408b8b1da7b8b15b4263: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260618 22:33:31.808079 25928 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260618 22:33:31.810202 25742 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 863af7c92037482dbd2d470608c0a984 (127.18.150.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:31.810845 25792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 863af7c92037482dbd2d470608c0a984 (127.18.150.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:31.813928 25742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:31.814590 25820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:31.814812 25820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:33:31.815177 25742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:31.815412 25932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:31.815909 25932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:31.834439 25874 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 FOLLOWER]: Refusing update from remote peer 2b9615ea036b408b8b1da7b8b15b4263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20260618 22:33:31.839210 25168 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 -> Peer 0857f29096284b568f1731395c320ab1 (127.18.150.188:34705): Couldn't send request to peer 0857f29096284b568f1731395c320ab1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260618 22:33:31.840859 25746 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:31.841226 25746 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:31.841413 25746 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: stopping tablet replica
I20260618 22:33:31.846864 25746 master.cc:552] Master@127.18.150.188:34705 shutdown complete.
I20260618 22:33:31.846928 25746 master_runner.cc:305] Clearing existing system tablet
I20260618 22:33:31.847210 25746 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:31.847316 25746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:31.849332 25746 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260618 22:33:31.850597 25935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:31.850880 25746 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:31.851125 25746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260618 22:33:31.851214 25746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/wal
uuid: "0857f29096284b568f1731395c320ab1"
format_stamp: "Formatted at 2026-06-18 22:33:31 on dist-test-slave-h7jz"
server_key: "f0e91c9c81736a9cfea9d2dcd894e478"
server_key_iv: "1bf0affdf4c3143ea09535f37ccc1175"
server_key_version: "encryptionkey@0"
I20260618 22:33:31.854439 25746 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:31.858804 25746 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260618 22:33:31.858932 25746 log.cc:1196] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260618 22:33:31.859288 25746 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: Deleting consensus metadata
I20260618 22:33:31.859488 25746 master_runner.cc:315] Copying system tablet from 127.18.150.190:35785
I20260618 22:33:31.861583 25746 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: tablet copy: Beginning tablet copy session from remote peer at address 127.18.150.190:35785
I20260618 22:33:31.876791 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.862312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.18.150.190:35785)
0618 22:33:31.862543 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:31.862550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:31.866452 (+ 3902us) server_negotiation.cc:407] Connection header received
0618 22:33:31.866636 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:31.866642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:31.866710 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:31.866797 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:31.867570 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.868285 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.869480 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.869904 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.870073 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.871771 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:31.871776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:31.871779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:31.871809 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:31.874411 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.875004 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.875009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.875013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.875125 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:31.875479 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:31.875485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:31.875488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:31.875762 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:31.875883 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:31.876250 (+ 367us) server_negotiation.cc:299] Negotiation successful
0618 22:33:31.876445 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260618 22:33:31.876847 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:31.862279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:60650)
0618 22:33:31.866338 (+ 4059us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:31.866355 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:31.866506 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:31.866830 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:31.866835 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:31.867208 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:31.867423 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.867432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.868415 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.868421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.869358 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:31.869373 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:31.870082 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:31.870088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:31.870289 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:31.870890 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:31.870909 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:31.871579 (+ 670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:31.874563 (+ 2984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.874569 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.874577 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.874896 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.875224 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:31.875229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:31.875233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:31.875382 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:31.875865 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:31.875869 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:31.876019 (+ 150us) client_negotiation.cc:769] Sending connection context
0618 22:33:31.876130 (+ 111us) client_negotiation.cc:240] Negotiation successful
0618 22:33:31.876443 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3910,"thread_start_us":3808,"threads_started":1}
I20260618 22:33:31.877600 25204 tablet_copy_service.cc:140] P 2b9615ea036b408b8b1da7b8b15b4263: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0857f29096284b568f1731395c320ab1 ({username='oryx', principal='oryx/127.18.150.188@KRBTEST.COM'} at 127.0.0.1:60650)
I20260618 22:33:31.877745 25204 tablet_copy_service.cc:161] P 2b9615ea036b408b8b1da7b8b15b4263: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0857f29096284b568f1731395c320ab1 at {username='oryx', principal='oryx/127.18.150.188@KRBTEST.COM'} at 127.0.0.1:60650: session id = 0857f29096284b568f1731395c320ab1-00000000000000000000000000000000
I20260618 22:33:31.880232 25204 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263: Tablet Copy: opened 0 blocks and 1 log segments
I20260618 22:33:31.881867 25746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260618 22:33:31.886655 25746 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: tablet copy: Starting download of 0 data blocks...
I20260618 22:33:31.886901 25746 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: tablet copy: Starting download of 1 WAL segments...
I20260618 22:33:31.888860 25746 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260618 22:33:31.891850 25204 tablet_copy_service.cc:342] P 2b9615ea036b408b8b1da7b8b15b4263: Request end of tablet copy session 0857f29096284b568f1731395c320ab1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.150.188@KRBTEST.COM'} at 127.0.0.1:60650
I20260618 22:33:31.891963 25204 tablet_copy_service.cc:433] P 2b9615ea036b408b8b1da7b8b15b4263: ending tablet copy session 0857f29096284b568f1731395c320ab1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0857f29096284b568f1731395c320ab1
I20260618 22:33:31.902594 25746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:31.905222 25942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:31.906260 25943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:31.913710 25945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:31.918051 25746 server_base.cc:1034] running on GCE node
I20260618 22:33:31.918383 25746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:31.918746 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 25245
I20260618 22:33:31.919446 25746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:31.922724 25746 hybrid_clock.cc:648] HybridClock initialized: now 1781822011922675 us; error 37 us; skew 500 ppm
I20260618 22:33:31.922983 25746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:31 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822011, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:31.925138 25746 init.cc:377] Logged in from keytab as oryx/127.18.150.188@KRBTEST.COM (short username oryx)
I20260618 22:33:31.935035 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:43405
--local_ip_for_outbound_sockets=127.18.150.129
--tserver_master_addrs=127.18.150.190:35785
--webserver_port=44433
--webserver_interface=127.18.150.129
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab
--principal=oryx/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.150.190:35785,127.18.150.189:37051
--tserver_master_addrs=127.18.150.190:35785,127.18.150.189:37051,127.18.150.188:34705 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:33:31.949671 25746 webserver.cc:460] Webserver started at http://127.18.150.188:38399/ using document root <none> and password file <none>
I20260618 22:33:31.950078 25746 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:31.950184 25746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:31.953276 25746 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260618 22:33:31.955284 25953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:31.955722 25746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260618 22:33:31.955849 25746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/wal
uuid: "0857f29096284b568f1731395c320ab1"
format_stamp: "Formatted at 2026-06-18 22:33:31 on dist-test-slave-h7jz"
server_key: "f0e91c9c81736a9cfea9d2dcd894e478"
server_key_iv: "1bf0affdf4c3143ea09535f37ccc1175"
server_key_version: "encryptionkey@0"
I20260618 22:33:31.956019 25746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:31.980068 25746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:31.984392 25746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:32.019974 25746 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.188:34705
I20260618 22:33:32.020037 26004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.188:34705 every 8 connection(s)
I20260618 22:33:32.020936 25746 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
I20260618 22:33:32.026119 26005 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:33:32.029706 26005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: Bootstrap starting.
I20260618 22:33:32.047186 26005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:33:32.047648 26005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: Bootstrap complete.
I20260618 22:33:32.048688 26005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } }
I20260618 22:33:32.048987 26005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0857f29096284b568f1731395c320ab1, State: Initialized, Role: LEARNER
I20260618 22:33:32.049278 26005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } }
I20260618 22:33:32.050099 26008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:32.050400 26005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:32.050565 26008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: This master's current role is: LEARNER
I20260618 22:33:32.059329 26019 catalog_manager.cc:1259] Loaded cluster ID: 6965aa89c6e8419c865b29d8b8ae8be6
I20260618 22:33:32.059428 26019 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: loading cluster ID for follower catalog manager: success
I20260618 22:33:32.062649 26019 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: acquiring CA information for follower catalog manager: success
I20260618 22:33:32.064991 26019 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260618 22:33:32.278935 25951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:32.279208 25951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:32.279335 25951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:32.286885 25951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:32.287000 25951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:32.287147 25951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:33:32.296619 25951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab
--principal=oryx/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:43405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=44433
--tserver_master_addrs=127.18.150.190:35785,127.18.150.189:37051,127.18.150.188:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:32.297925 25951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:32.299243 25951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:32.311818 26027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:32.311846 26025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:32.312048 26024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:32.312556 25951 server_base.cc:1034] running on GCE node
I20260618 22:33:32.313118 25951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:32.314015 25951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:32.315217 25951 hybrid_clock.cc:648] HybridClock initialized: now 1781822012315164 us; error 60 us; skew 500 ppm
I20260618 22:33:32.315502 25951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:32 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822012, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:32.318570 25951 init.cc:377] Logged in from keytab as oryx/127.18.150.129@KRBTEST.COM (short username oryx)
I20260618 22:33:32.319976 25951 webserver.cc:460] Webserver started at http://127.18.150.129:44433/ using document root <none> and password file <none>
I20260618 22:33:32.320670 25951 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:32.320797 25951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:32.326042 25951 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:33:32.328754 26034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:32.330096 25951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:33:32.330260 25951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "d3b54272febb41a292c6bf632baaedbc"
format_stamp: "Formatted at 2026-06-18 22:33:29 on dist-test-slave-h7jz"
server_key: "67fff952ec4ce2fcd1f1ca926895ec97"
server_key_iv: "760d9e0c0fca01150fb2ff7a2837d3be"
server_key_version: "encryptionkey@0"
I20260618 22:33:32.331176 25951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:32.351898 25951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:32.356163 25951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:32.356415 25951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:32.357425 25951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:32.358821 25951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:32.358911 25951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:32.359015 25951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:32.359081 25951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:32.390372 26127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.382052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:34705 (local address 127.0.0.1:49682)
0618 22:33:32.384452 (+ 2400us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:32.384492 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:32.384637 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:32.385087 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:32.385092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:32.385124 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:32.385335 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:32.385347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.386894 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.386901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:32.388476 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:32.388492 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.389566 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.389572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:32.389757 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:32.389790 (+ 33us) client_negotiation.cc:769] Sending connection context
0618 22:33:32.389876 (+ 86us) client_negotiation.cc:240] Negotiation successful
0618 22:33:32.390037 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2296,"thread_start_us":2207,"threads_started":1}
I20260618 22:33:32.390375 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.382250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49682 (local address 127.18.150.188:34705)
0618 22:33:32.382779 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:32.382786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:32.384597 (+ 1811us) server_negotiation.cc:407] Connection header received
0618 22:33:32.384783 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:32.384789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:32.384897 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:32.385045 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:32.385484 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.386762 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.388614 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.389399 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.389545 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:32.389749 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:32.389922 (+ 173us) server_negotiation.cc:299] Negotiation successful
0618 22:33:32.390065 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":220,"threads_started":1}
I20260618 22:33:32.391413 25980 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260618 22:33:32.392256 26008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b9615ea036b408b8b1da7b8b15b4263. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:32.392542 26008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: This master's current role is: LEARNER
I20260618 22:33:32.394711 25951 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:43405
I20260618 22:33:32.394773 26149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:43405 every 8 connection(s)
I20260618 22:33:32.396034 25951 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:33:32.399174 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 25951
I20260618 22:33:32.399520 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 25379
Jun 18 22:33:32 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822012, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.189@KRBTEST.COM
Jun 18 22:33:32 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822012, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.188@KRBTEST.COM
Jun 18 22:33:32 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822012, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
Jun 18 22:33:32 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.150.130@KRBTEST.COM
I20260618 22:33:32.429636 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.400178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:51445 (local address 127.18.150.188:34705)
0618 22:33:32.400946 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:32.400952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:32.403464 (+ 2512us) server_negotiation.cc:407] Connection header received
0618 22:33:32.404271 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:32.404277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:32.404381 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:32.404450 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:32.409358 (+ 4908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.410555 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.416039 (+ 5484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.416652 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.416791 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:32.422565 (+ 5774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:32.422573 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:32.422578 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:32.422617 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:32.425707 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:32.426866 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:32.426871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:32.426875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:32.427008 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:32.427528 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:32.427533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:32.427536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:32.427893 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:32.428004 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:32.429074 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0618 22:33:32.429299 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639}
I20260618 22:33:32.429664 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.398725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:49709 (local address 127.18.150.189:37051)
0618 22:33:32.411047 (+ 12322us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:32.411054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:32.411079 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:33:32.411140 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:32.411145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:32.411230 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:32.411339 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:32.412751 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.413920 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.415884 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.416512 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.416626 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:32.422054 (+ 5428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:32.422061 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:32.422066 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:32.422106 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:32.425693 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:32.426884 (+ 1191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:32.426889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:32.426892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:32.427008 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:32.427535 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:32.427540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:32.427543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:32.427883 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:32.428005 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:32.429074 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0618 22:33:32.429295 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12193,"thread_start_us":13862,"threads_started":1}
I20260618 22:33:32.431259 26156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.399889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:35919 (local address 127.18.150.190:35785)
0618 22:33:32.412719 (+ 12830us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:32.412725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:32.412744 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:33:32.412797 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:32.412802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:32.412870 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:32.412973 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:32.417026 (+ 4053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.418135 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.419913 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.420335 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.420443 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:32.424074 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:32.424080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:32.424085 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:32.424123 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:32.427697 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:32.428434 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:32.428440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:32.428443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:32.428588 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:32.429130 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:32.429135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:32.429139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:32.429485 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:32.429606 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:32.430654 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0618 22:33:32.430880 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12706,"thread_start_us":13239,"threads_started":1}
I20260618 22:33:32.432054 26157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.400160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:34705 (local address 127.18.150.129:51445)
0618 22:33:32.403378 (+ 3218us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:32.403394 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:32.403504 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:32.404951 (+ 1447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:32.404966 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:32.405884 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:32.406774 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:32.406800 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.410816 (+ 4016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.410823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:32.413026 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:32.413039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.417049 (+ 4010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.417055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:32.417387 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:32.419728 (+ 2341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:32.419755 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:32.422352 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:32.425855 (+ 3503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:32.425866 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:32.425886 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:32.426734 (+ 848us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:32.427131 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:32.427136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:32.427140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:32.427405 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:32.428085 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:32.428094 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:32.428628 (+ 534us) client_negotiation.cc:769] Sending connection context
0618 22:33:32.429910 (+ 1282us) client_negotiation.cc:240] Negotiation successful
0618 22:33:32.430538 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3107,"mutex_wait_us":199,"spinlock_wait_cycles":66688,"thread_start_us":2978,"threads_started":1}
I20260618 22:33:32.432197 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.399362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.129:35919)
0618 22:33:32.400199 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:32.400249 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:32.402628 (+ 2379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:32.414067 (+ 11439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:32.414072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:32.414545 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:32.415887 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:32.415897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.418280 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.418285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:32.419788 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:32.419806 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.420509 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.420515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:32.420701 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:32.421314 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:32.421335 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:32.423855 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:32.427863 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:32.427868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:32.427872 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:32.428291 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:32.428810 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:32.428815 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:32.428818 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:32.429001 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:32.429676 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:32.429681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:32.430235 (+ 554us) client_negotiation.cc:769] Sending connection context
0618 22:33:32.430363 (+ 128us) client_negotiation.cc:240] Negotiation successful
0618 22:33:32.431541 (+ 1178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":1951,"thread_start_us":4560,"threads_started":1}
I20260618 22:33:32.432087 26158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.400553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:37051 (local address 127.18.150.129:49709)
0618 22:33:32.403912 (+ 3359us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:32.403927 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:32.404034 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:32.411527 (+ 7493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:32.411534 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:32.412157 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:32.412486 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:32.412498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.414097 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.414102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:32.415703 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:32.415717 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.416941 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.416949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:32.417381 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:32.418272 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:32.418302 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:32.421470 (+ 3168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:32.425855 (+ 4385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:32.425866 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:32.425886 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:32.426764 (+ 878us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:32.427131 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:32.427136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:32.427140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:32.427405 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:32.428085 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:32.428094 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:32.428628 (+ 534us) client_negotiation.cc:769] Sending connection context
0618 22:33:32.429910 (+ 1282us) client_negotiation.cc:240] Negotiation successful
0618 22:33:32.430657 (+ 747us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3257,"mutex_wait_us":318,"spinlock_wait_cycles":13312,"thread_start_us":3335,"threads_started":1}
I20260618 22:33:32.434026 26151 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35785
I20260618 22:33:32.434068 26152 heartbeater.cc:347] Connected to a master server at 127.18.150.189:37051
I20260618 22:33:32.434020 26150 heartbeater.cc:347] Connected to a master server at 127.18.150.188:34705
I20260618 22:33:32.434376 26151 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:32.434433 26152 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:32.434373 26150 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:32.435406 26151 heartbeater.cc:511] Master 127.18.150.190:35785 requested a full tablet report, sending...
I20260618 22:33:32.435458 26150 heartbeater.cc:511] Master 127.18.150.188:34705 requested a full tablet report, sending...
I20260618 22:33:32.435940 26152 heartbeater.cc:511] Master 127.18.150.189:37051 requested a full tablet report, sending...
I20260618 22:33:32.436450 26160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.401442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32804 (local address 127.18.150.130:37773)
0618 22:33:32.415274 (+ 13832us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:32.415282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:32.415309 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:33:32.415375 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:32.415382 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:32.415660 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:32.415812 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:32.418446 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.419481 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.422462 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:32.423887 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:32.423992 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:32.429831 (+ 5839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:32.429849 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:32.429868 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:32.429906 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:32.433291 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:32.434179 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:32.434187 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:32.434190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:32.434353 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:32.434770 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:32.434776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:32.434779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:32.435191 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:32.435458 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:32.435816 (+ 358us) server_negotiation.cc:299] Negotiation successful
0618 22:33:32.436111 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13668,"thread_start_us":13519,"threads_started":1}
I20260618 22:33:32.437136 25184 ts_manager.cc:194] Re-registered known tserver with Master: d3b54272febb41a292c6bf632baaedbc (127.18.150.129:43405)
I20260618 22:33:32.437845 25502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:32.437879 25970 ts_manager.cc:194] Registered new tserver with Master: d3b54272febb41a292c6bf632baaedbc (127.18.150.129:43405)
I20260618 22:33:32.437915 25184 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:35919
I20260618 22:33:32.440194 25782 ts_manager.cc:194] Registered new tserver with Master: d3b54272febb41a292c6bf632baaedbc (127.18.150.129:43405)
I20260618 22:33:32.519014 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 25379
I20260618 22:33:32.531397 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:37773
--local_ip_for_outbound_sockets=127.18.150.130
--tserver_master_addrs=127.18.150.190:35785
--webserver_port=36235
--webserver_interface=127.18.150.130
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab
--principal=oryx/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.150.190:35785,127.18.150.189:37051
--tserver_master_addrs=127.18.150.190:35785,127.18.150.189:37051,127.18.150.188:34705 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:32.840318 26162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:32.840607 26162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:32.840739 26162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:32.848060 26162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:32.848188 26162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:32.848332 26162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:33:32.858139 26162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab
--principal=oryx/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:37773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=36235
--tserver_master_addrs=127.18.150.190:35785,127.18.150.189:37051,127.18.150.188:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:32.859438 26162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:32.860697 26162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:33:32.864137 26123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: NON_VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: true } } }
I20260618 22:33:32.864530 26123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:32.865715 26167 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263: attempting to promote NON_VOTER 0857f29096284b568f1731395c320ab1 to VOTER
I20260618 22:33:32.866408 26167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } }
I20260618 22:33:32.869019 25980 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 LEARNER]: Refusing update from remote peer 2b9615ea036b408b8b1da7b8b15b4263: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260618 22:33:32.869123 25792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 FOLLOWER]: Refusing update from remote peer 2b9615ea036b408b8b1da7b8b15b4263: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260618 22:33:32.869659 26166 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260618 22:33:32.870152 26123 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260618 22:33:32.877741 26171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:32.878834 26123 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0857f29096284b568f1731395c320ab1 (127.18.150.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:32.879791 25980 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0857f29096284b568f1731395c320ab1 (127.18.150.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:32.880111 25792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0857f29096284b568f1731395c320ab1 (127.18.150.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:32.881844 26162 server_base.cc:1034] running on GCE node
W20260618 22:33:32.882395 26172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:32.882833 26176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:32.882800 26167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:32.883059 26167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:32.883311 26167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:32.883508 26167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:32.883747 26167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:32.883950 26167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:32.884038 26162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:32.884510 26008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:32.884768 26008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:33:32.884711 26178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b9615ea036b408b8b1da7b8b15b4263" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:32.885051 26178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:33:32.885697 26162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:32.886896 26162 hybrid_clock.cc:648] HybridClock initialized: now 1781822012886830 us; error 54 us; skew 500 ppm
I20260618 22:33:32.887205 26162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:32 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822012, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:32.890300 26162 init.cc:377] Logged in from keytab as oryx/127.18.150.130@KRBTEST.COM (short username oryx)
I20260618 22:33:32.892030 26162 webserver.cc:460] Webserver started at http://127.18.150.130:36235/ using document root <none> and password file <none>
I20260618 22:33:32.892727 26162 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:32.892850 26162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:32.898538 26162 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:33:32.901213 26184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:32.902660 26162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:32.902822 26162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "d7632b2891444502858ae543d072076e"
format_stamp: "Formatted at 2026-06-18 22:33:29 on dist-test-slave-h7jz"
server_key: "6f3efe3dd8a5048b1b702705b3327058"
server_key_iv: "6e821e39b292313c8c523bef67359665"
server_key_version: "encryptionkey@0"
I20260618 22:33:32.903923 26162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:32.939050 26162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:32.943264 26162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:32.943532 26162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:32.944537 26162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:32.945926 26162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:32.946053 26162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:32.946158 26162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:32.946233 26162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:32.975915 26162 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:37773
I20260618 22:33:32.975965 26296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:37773 every 8 connection(s)
I20260618 22:33:32.977367 26162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:33:32.980360 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 26162
I20260618 22:33:32.980656 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 25512
Jun 18 22:33:33 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.150.131@KRBTEST.COM
I20260618 22:33:33.026589 26306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.982001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46978 (local address 127.18.150.131:38539)
0618 22:33:33.004223 (+ 22222us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.004233 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.005174 (+ 941us) server_negotiation.cc:407] Connection header received
0618 22:33:33.005493 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.005504 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.005858 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.006190 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:33.008490 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.010023 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.011856 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.012722 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.012838 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.021178 (+ 8340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:33.021192 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:33.021205 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:33.021245 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:33.023903 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.024582 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.024589 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.024592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.024718 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.025063 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.025068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.025071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.025400 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:33.025668 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.025991 (+ 323us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.026249 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21997,"thread_start_us":22260,"threads_started":1}
Jun 18 22:33:33 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822012, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.189@KRBTEST.COM
I20260618 22:33:33.028076 25635 generic_service.cc:196] Checking for leaks (request via RPC)
Jun 18 22:33:33 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822012, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
Jun 18 22:33:33 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822012, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.188@KRBTEST.COM
I20260618 22:33:33.037961 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.986921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:56319 (local address 127.18.150.188:34705)
0618 22:33:33.010258 (+ 23337us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.010266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.010941 (+ 675us) server_negotiation.cc:407] Connection header received
0618 22:33:33.011005 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.011010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.011092 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.011184 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:33.013421 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.014397 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.016921 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.017548 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.017660 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.029858 (+ 12198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:33.029866 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:33.029870 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:33.029911 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:33.033451 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.034667 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.034673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.034676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.034821 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.035359 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.035364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.035367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.035697 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:33.035828 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.037230 (+ 1402us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.037476 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23198,"thread_start_us":23098,"threads_started":1}
I20260618 22:33:33.038722 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.986464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:34705 (local address 127.18.150.130:56319)
0618 22:33:33.010692 (+ 24228us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:33.010714 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:33.010822 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:33.011361 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:33.011365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:33.012248 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:33.013253 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.013274 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.014536 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.014541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.016765 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.016779 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.017896 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.017902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.018235 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.018851 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:33.018876 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:33.029614 (+ 10738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:33.033632 (+ 4018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.033641 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.033660 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.034512 (+ 852us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.034949 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.034954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.034958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.035226 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.035898 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:33.035906 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:33.036369 (+ 463us) client_negotiation.cc:769] Sending connection context
0618 22:33:33.036671 (+ 302us) client_negotiation.cc:240] Negotiation successful
0618 22:33:33.037252 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24098,"mutex_wait_us":210,"spinlock_wait_cycles":18304,"thread_start_us":23952,"threads_started":1}
I20260618 22:33:33.040191 26297 heartbeater.cc:347] Connected to a master server at 127.18.150.188:34705
I20260618 22:33:33.040509 26297 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:33.041361 26297 heartbeater.cc:511] Master 127.18.150.188:34705 requested a full tablet report, sending...
I20260618 22:33:33.041471 26304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.981147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:36453 (local address 127.18.150.190:35785)
0618 22:33:33.004561 (+ 23414us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.004568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.006460 (+ 1892us) server_negotiation.cc:407] Connection header received
0618 22:33:33.010528 (+ 4068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.010535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.010657 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.010743 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:33.013997 (+ 3254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.015035 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.016962 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.017365 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.017472 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.035080 (+ 17608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:33.035089 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:33.035094 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:33.035151 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:33.038616 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.039403 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.039408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.039411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.039548 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.039995 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.040000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.040003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.040324 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:33.040455 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.040907 (+ 452us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.041115 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23273,"thread_start_us":24382,"threads_started":1}
I20260618 22:33:33.041679 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.980818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.130:36453)
0618 22:33:33.006295 (+ 25477us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:33.006358 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:33.009136 (+ 2778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:33.011251 (+ 2115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:33.011262 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:33.012248 (+ 986us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:33.013830 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.013842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.015163 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.015169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.016831 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.016847 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.017577 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.017584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.017882 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.018640 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:33.018672 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:33.034776 (+ 16104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:33.038754 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.038759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.038763 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.039287 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.039702 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.039707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.039711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.039880 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.040440 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:33.040445 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:33.040633 (+ 188us) client_negotiation.cc:769] Sending connection context
0618 22:33:33.040779 (+ 146us) client_negotiation.cc:240] Negotiation successful
0618 22:33:33.041185 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25119,"mutex_wait_us":241,"spinlock_wait_cycles":13440,"thread_start_us":28630,"threads_started":1}
I20260618 22:33:33.042515 26298 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35785
I20260618 22:33:33.042658 26298 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:33.043040 26302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.980818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:37051 (local address 127.18.150.130:40433)
0618 22:33:33.009926 (+ 29108us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:33.009976 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:33.010103 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:33.011248 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:33.011261 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:33.012259 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:33.013545 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.013555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.014983 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.014988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.016767 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.016779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.017915 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.017920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.018239 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.018946 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:33.018964 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:33.029142 (+ 10178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:33.033632 (+ 4490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.033653 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.033660 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.034512 (+ 852us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.034982 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.034986 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.034989 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.035230 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.035905 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:33.035909 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:33.038565 (+ 2656us) client_negotiation.cc:769] Sending connection context
0618 22:33:33.040025 (+ 1460us) client_negotiation.cc:240] Negotiation successful
0618 22:33:33.042305 (+ 2280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28894,"mutex_wait_us":2774,"spinlock_wait_cycles":116224,"thread_start_us":24826,"threads_started":1}
I20260618 22:33:33.043219 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:32.980222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:40433 (local address 127.18.150.189:37051)
0618 22:33:33.005396 (+ 25174us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.005401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.010704 (+ 5303us) server_negotiation.cc:407] Connection header received
0618 22:33:33.010778 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.010783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.010857 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.010956 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:33.013707 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.014847 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.016936 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.017547 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.017660 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.029926 (+ 12266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:33.029933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:33.029967 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:33.030018 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:33.033452 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.034674 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.034679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.034682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.034873 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.035359 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.035364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.035367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.035713 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:33.035997 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.042681 (+ 6684us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.042901 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24880,"thread_start_us":24978,"threads_started":1}
I20260618 22:33:33.043254 25970 ts_manager.cc:194] Registered new tserver with Master: d7632b2891444502858ae543d072076e (127.18.150.130:37773)
I20260618 22:33:33.043100 26298 heartbeater.cc:511] Master 127.18.150.190:35785 requested a full tablet report, sending...
I20260618 22:33:33.044446 25184 ts_manager.cc:194] Re-registered known tserver with Master: d7632b2891444502858ae543d072076e (127.18.150.130:37773)
I20260618 22:33:33.044467 26299 heartbeater.cc:347] Connected to a master server at 127.18.150.189:37051
I20260618 22:33:33.044586 26299 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:33.044996 26299 heartbeater.cc:511] Master 127.18.150.189:37051 requested a full tablet report, sending...
I20260618 22:33:33.045311 25184 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:36453
I20260618 22:33:33.046119 25782 ts_manager.cc:194] Registered new tserver with Master: d7632b2891444502858ae543d072076e (127.18.150.130:37773)
I20260618 22:33:33.125175 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 25512
I20260618 22:33:33.137470 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:38539
--local_ip_for_outbound_sockets=127.18.150.131
--tserver_master_addrs=127.18.150.190:35785
--webserver_port=37527
--webserver_interface=127.18.150.131
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab
--principal=oryx/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.150.190:35785,127.18.150.189:37051
--tserver_master_addrs=127.18.150.190:35785,127.18.150.189:37051,127.18.150.188:34705 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:33:33.441043 26151 heartbeater.cc:503] Master 127.18.150.190:35785 was elected leader, sending a full tablet report...
W20260618 22:33:33.452754 26310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:33.453148 26310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:33.453307 26310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:33.460902 26310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:33.461014 26310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:33.461156 26310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:33:33.470944 26310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab
--principal=oryx/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:38539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=37527
--tserver_master_addrs=127.18.150.190:35785,127.18.150.189:37051,127.18.150.188:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:33.472296 26310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:33.473613 26310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:33.486395 26315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:33.486366 26316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:33.486342 26318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:33.487180 26310 server_base.cc:1034] running on GCE node
I20260618 22:33:33.487751 26310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:33.488629 26310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:33.489851 26310 hybrid_clock.cc:648] HybridClock initialized: now 1781822013489803 us; error 41 us; skew 500 ppm
I20260618 22:33:33.490187 26310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:33 dist-test-slave-h7jz krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822013, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:33.493093 26310 init.cc:377] Logged in from keytab as oryx/127.18.150.131@KRBTEST.COM (short username oryx)
I20260618 22:33:33.494434 26310 webserver.cc:460] Webserver started at http://127.18.150.131:37527/ using document root <none> and password file <none>
I20260618 22:33:33.495106 26310 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:33.495213 26310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:33.500375 26310 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:33:33.502928 26325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:33.504237 26310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:33.504400 26310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "f6a71f206db54fe4afde3cc0415e0ba6"
format_stamp: "Formatted at 2026-06-18 22:33:30 on dist-test-slave-h7jz"
server_key: "289b2f5b0ced76174b22abce3e5b7a77"
server_key_iv: "fe8f22086e58fc19520cb8b4f86a8816"
server_key_version: "encryptionkey@0"
I20260618 22:33:33.505378 26310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:33.519910 26310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:33.523947 26310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:33.524206 26310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:33.525182 26310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:33.526515 26310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:33.526607 26310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:33.526703 26310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:33.526768 26310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:33.555660 26310 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:38539
I20260618 22:33:33.555699 26437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:38539 every 8 connection(s)
I20260618 22:33:33.557113 26310 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:33:33.566179 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 26310
Jun 18 22:33:33 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822013, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
Jun 18 22:33:33 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822013, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.189@KRBTEST.COM
Jun 18 22:33:33 dist-test-slave-h7jz krb5kdc[25139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822013, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.188@KRBTEST.COM
I20260618 22:33:33.599121 26443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.560116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:50657 (local address 127.18.150.190:35785)
0618 22:33:33.578478 (+ 18362us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.578484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.578501 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:33:33.580301 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.580306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.580383 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.580528 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:33.581748 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.582786 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.584769 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.585367 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.585477 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.591524 (+ 6047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:33.591530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:33.591535 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:33.591570 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:33.595106 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.596006 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.596012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.596015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.596182 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.596728 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.596733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.596737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.597082 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:33.597200 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.598491 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.598738 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18253,"thread_start_us":18168,"threads_started":1}
I20260618 22:33:33.601341 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.560905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:37887 (local address 127.18.150.189:37051)
0618 22:33:33.570367 (+ 9462us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.570374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.577881 (+ 7507us) server_negotiation.cc:407] Connection header received
0618 22:33:33.578478 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.578484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.578580 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.578642 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:33.580914 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.582080 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.584939 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.585621 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.585734 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.591478 (+ 5744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:33.591486 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:33.591491 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:33.591543 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:33.597705 (+ 6162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.598953 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.598959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.598963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.599108 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.599832 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.599837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.599840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.600167 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:33.600296 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.600794 (+ 498us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.600995 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9307,"thread_start_us":8959,"threads_started":1}
I20260618 22:33:33.601446 26444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.560465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.131:50657)
0618 22:33:33.575424 (+ 14959us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:33.575489 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:33.580141 (+ 4652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:33.580613 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:33.580617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:33.581271 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:33.581484 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.581558 (+ 74us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.582904 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.582910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.584642 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.584656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.586053 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.586060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.586500 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.587484 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:33.587513 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:33.590562 (+ 3049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:33.595271 (+ 4709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.595279 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.595298 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.595864 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.596325 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.596330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.596333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.596601 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.597304 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:33.597312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:33.597753 (+ 441us) client_negotiation.cc:769] Sending connection context
0618 22:33:33.599434 (+ 1681us) client_negotiation.cc:240] Negotiation successful
0618 22:33:33.600012 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14533,"mutex_wait_us":177,"spinlock_wait_cycles":3840,"thread_start_us":16508,"threads_started":1}
I20260618 22:33:33.602666 26445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.561055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:37051 (local address 127.18.150.131:37887)
0618 22:33:33.577750 (+ 16695us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:33.577775 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:33.578137 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:33.579061 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:33.579070 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:33.579780 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:33.580715 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.580740 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.582247 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.582252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.584810 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.584821 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.586053 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.586060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.586516 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.587649 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:33.587699 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:33.591293 (+ 3594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:33.598128 (+ 6835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.598137 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.598140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.598828 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.599550 (+ 722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.599554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.599557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.599708 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.600369 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:33.600372 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:33.600515 (+ 143us) client_negotiation.cc:769] Sending connection context
0618 22:33:33.602077 (+ 1562us) client_negotiation.cc:240] Negotiation successful
0618 22:33:33.602334 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16577,"mutex_wait_us":394,"thread_start_us":15115,"threads_started":1}
I20260618 22:33:33.603122 26439 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35785
I20260618 22:33:33.603435 26439 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:33.604099 26439 heartbeater.cc:511] Master 127.18.150.190:35785 requested a full tablet report, sending...
I20260618 22:33:33.605548 26440 heartbeater.cc:347] Connected to a master server at 127.18.150.189:37051
I20260618 22:33:33.605670 26440 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:33.605674 25184 ts_manager.cc:194] Re-registered known tserver with Master: f6a71f206db54fe4afde3cc0415e0ba6 (127.18.150.131:38539)
I20260618 22:33:33.606154 26440 heartbeater.cc:511] Master 127.18.150.189:37051 requested a full tablet report, sending...
I20260618 22:33:33.606804 25184 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:50657
I20260618 22:33:33.607453 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.560465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:34705 (local address 127.18.150.131:55405)
0618 22:33:33.576036 (+ 15571us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:33.576053 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:33.578153 (+ 2100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:33.579064 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:33.579071 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:33.579833 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:33.580719 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.580743 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.582359 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.582368 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.584640 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.584655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.586053 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.586060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.586514 (+ 454us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.587487 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:33.587524 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:33.591579 (+ 4055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:33.597853 (+ 6274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.597857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.597860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.598472 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.605586 (+ 7114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:33.605590 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:33.605593 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:33.605763 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:33.606505 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:33.606510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:33.606651 (+ 141us) client_negotiation.cc:769] Sending connection context
0618 22:33:33.606765 (+ 114us) client_negotiation.cc:240] Negotiation successful
0618 22:33:33.607042 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15430,"mutex_wait_us":826,"spinlock_wait_cycles":43904,"thread_start_us":16800,"threads_started":1}
I20260618 22:33:33.607540 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.559832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:55405 (local address 127.18.150.188:34705)
0618 22:33:33.560263 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.560269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.576153 (+ 15884us) server_negotiation.cc:407] Connection header received
0618 22:33:33.578505 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.578512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.578611 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.578691 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:33.580924 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.582141 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.584963 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.585560 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.585651 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.591770 (+ 6119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:33.591777 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:33.591782 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:33.591820 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:33.596048 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.605152 (+ 9104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.605158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.605161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.605343 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.605900 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.605905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.605909 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.606296 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:33.606433 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.606918 (+ 485us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.607212 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":370,"threads_started":1}
I20260618 22:33:33.608042 25782 ts_manager.cc:194] Registered new tserver with Master: f6a71f206db54fe4afde3cc0415e0ba6 (127.18.150.131:38539)
I20260618 22:33:33.613130 26438 heartbeater.cc:347] Connected to a master server at 127.18.150.188:34705
I20260618 22:33:33.613281 26438 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:33.613628 26438 heartbeater.cc:511] Master 127.18.150.188:34705 requested a full tablet report, sending...
I20260618 22:33:33.614791 25970 ts_manager.cc:194] Registered new tserver with Master: f6a71f206db54fe4afde3cc0415e0ba6 (127.18.150.131:38539)
I20260618 22:33:33.628543 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.588171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49696 (local address 127.18.150.188:34705)
0618 22:33:33.607716 (+ 19545us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.607721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.614336 (+ 6615us) server_negotiation.cc:407] Connection header received
0618 22:33:33.614643 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.614648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.614738 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.614906 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:33.616889 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.617616 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.619002 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.619394 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.619498 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.623054 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:33.623061 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:33.623066 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:33.623104 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:33.625567 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.626389 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.626395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.626399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.626550 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.626921 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.626926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.626930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.627271 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:33.627501 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.628073 (+ 572us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.628310 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19466,"thread_start_us":24061,"threads_started":1}
I20260618 22:33:33.634295 26443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.587544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60654 (local address 127.18.150.190:35785)
0618 22:33:33.599376 (+ 11832us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.599381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.620464 (+ 21083us) server_negotiation.cc:407] Connection header received
0618 22:33:33.620618 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.620624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.620720 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.620829 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:33.623010 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.624090 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.625850 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.626315 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.626443 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.629066 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:33.629072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:33.629077 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:33.629119 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:33.631968 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.632536 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.632542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.632546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.632657 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.632982 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.632987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.632990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.633258 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:33.633475 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.633767 (+ 292us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.634008 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11733,"thread_start_us":27619,"threads_started":1}
I20260618 22:33:33.636439 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.587854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53298 (local address 127.18.150.189:37051)
0618 22:33:33.602040 (+ 14186us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.602045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.622596 (+ 20551us) server_negotiation.cc:407] Connection header received
0618 22:33:33.622739 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.622745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.622837 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.622938 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:33.624589 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.625631 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.627329 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.628004 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.628113 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.631306 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:33.631310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:33.631314 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:33.631389 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:33.633621 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.634399 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.634403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.634406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.634547 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:33.634911 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:33.634914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:33.634916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:33.635196 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:33.635410 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.635787 (+ 377us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.636111 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14050,"thread_start_us":25319,"threads_started":1}
I20260618 22:33:33.638376 25184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:33:33.644922 25184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:33:33.673434 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.663170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.18.150.129:43405)
0618 22:33:33.666371 (+ 3201us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.666381 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.666414 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:33:33.666523 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.666531 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.666882 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.667080 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:33.667583 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.669113 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.671042 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.672384 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.672557 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.672732 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.672959 (+ 227us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.673120 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3029,"thread_start_us":2934,"threads_started":1}
I20260618 22:33:33.673630 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.662352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43405 (local address 127.0.0.1:43062)
0618 22:33:33.663114 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:33.663140 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:33.663268 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:33.667068 (+ 3800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:33.667073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:33.667099 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:33.667446 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.667457 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.669249 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.669255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.670904 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.670916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.672588 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.672597 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.672769 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.672809 (+ 40us) client_negotiation.cc:769] Sending connection context
0618 22:33:33.672909 (+ 100us) client_negotiation.cc:240] Negotiation successful
0618 22:33:33.673159 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":84,"thread_start_us":1588,"threads_started":1}
I20260618 22:33:33.674780 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.663497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.18.150.130:37773)
0618 22:33:33.664054 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.664065 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.666892 (+ 2827us) server_negotiation.cc:407] Connection header received
0618 22:33:33.667180 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.667189 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.667446 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.667601 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:33.668265 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.669674 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.672138 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.673430 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.673654 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.673834 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.674359 (+ 525us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.674523 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":254,"threads_started":1}
I20260618 22:33:33.675817 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.662352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:37773 (local address 127.0.0.1:32810)
0618 22:33:33.666775 (+ 4423us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:33.666795 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:33.667001 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:33.667813 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:33.667816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:33.667834 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:33.668083 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.668090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.669838 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.669847 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.671980 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.671995 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.674017 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.674024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.674185 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.674211 (+ 26us) client_negotiation.cc:769] Sending connection context
0618 22:33:33.674294 (+ 83us) client_negotiation.cc:240] Negotiation successful
0618 22:33:33.675467 (+ 1173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4307,"mutex_wait_us":1718,"thread_start_us":278,"threads_started":1}
I20260618 22:33:33.677323 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.662669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:38539 (local address 127.0.0.1:46984)
0618 22:33:33.664619 (+ 1950us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:33.664638 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:33.664762 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:33.670066 (+ 5304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:33.670072 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:33.670107 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:33.670524 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.670537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.672548 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.672554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.674675 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:33.674689 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.676542 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.676548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:33.676721 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.676753 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:33:33.676831 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:33:33.676956 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1827,"mutex_wait_us":16,"thread_start_us":3832,"threads_started":1}
I20260618 22:33:33.677343 26468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:33.663148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46984 (local address 127.18.150.131:38539)
0618 22:33:33.669451 (+ 6303us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:33.669460 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:33.669488 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:33:33.669567 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:33.669574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:33.669810 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:33.670728 (+ 918us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:33.671063 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.672421 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.675148 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:33.676353 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:33.676505 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:33.676676 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:33.676889 (+ 213us) server_negotiation.cc:299] Negotiation successful
0618 22:33:33.677054 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6135,"thread_start_us":4249,"threads_started":1}
I20260618 22:33:33.678109 26082 tablet_service.cc:1467] Processing CreateTablet for tablet ba6280d444ca4d6eb68c4b343d70a8ad (DEFAULT_TABLE table=test-table [id=9bd875589d884646a5acea7c4da3afcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:33.679567 26082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6280d444ca4d6eb68c4b343d70a8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:33.681365 26373 tablet_service.cc:1467] Processing CreateTablet for tablet ba6280d444ca4d6eb68c4b343d70a8ad (DEFAULT_TABLE table=test-table [id=9bd875589d884646a5acea7c4da3afcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:33.682812 26373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6280d444ca4d6eb68c4b343d70a8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:33.684118 26232 tablet_service.cc:1467] Processing CreateTablet for tablet ba6280d444ca4d6eb68c4b343d70a8ad (DEFAULT_TABLE table=test-table [id=9bd875589d884646a5acea7c4da3afcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:33.685441 26232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6280d444ca4d6eb68c4b343d70a8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:33.699486 26473 tablet_bootstrap.cc:492] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: Bootstrap starting.
I20260618 22:33:33.701570 26472 tablet_bootstrap.cc:492] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: Bootstrap starting.
I20260618 22:33:33.702730 26474 tablet_bootstrap.cc:492] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: Bootstrap starting.
I20260618 22:33:33.703430 26473 tablet_bootstrap.cc:654] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:33.704798 26472 tablet_bootstrap.cc:654] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:33.704800 26473 log.cc:824] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:33.706095 26472 log.cc:824] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:33.706580 26474 tablet_bootstrap.cc:654] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:33.707875 26474 log.cc:824] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:33.708565 26473 tablet_bootstrap.cc:492] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: No bootstrap required, opened a new log
I20260618 22:33:33.708952 26473 ts_tablet_manager.cc:1397] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260618 22:33:33.710891 26474 tablet_bootstrap.cc:492] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: No bootstrap required, opened a new log
I20260618 22:33:33.711107 26472 tablet_bootstrap.cc:492] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: No bootstrap required, opened a new log
I20260618 22:33:33.711131 26474 ts_tablet_manager.cc:1397] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260618 22:33:33.711472 26472 ts_tablet_manager.cc:1397] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260618 22:33:33.718577 26474 raft_consensus.cc:348] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } }
I20260618 22:33:33.718986 26474 raft_consensus.cc:374] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:33.719096 26474 raft_consensus.cc:729] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a71f206db54fe4afde3cc0415e0ba6, State: Initialized, Role: FOLLOWER
I20260618 22:33:33.719246 26473 raft_consensus.cc:348] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } }
I20260618 22:33:33.719668 26473 raft_consensus.cc:374] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:33.719769 26473 raft_consensus.cc:729] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7632b2891444502858ae543d072076e, State: Initialized, Role: FOLLOWER
I20260618 22:33:33.719681 26474 consensus_queue.cc:260] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } }
I20260618 22:33:33.720315 26473 consensus_queue.cc:260] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } }
I20260618 22:33:33.721447 26474 ts_tablet_manager.cc:1428] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260618 22:33:33.721778 26439 heartbeater.cc:503] Master 127.18.150.190:35785 was elected leader, sending a full tablet report...
I20260618 22:33:33.721519 26472 raft_consensus.cc:348] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } }
I20260618 22:33:33.725792 26472 raft_consensus.cc:374] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:33.725915 26472 raft_consensus.cc:729] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3b54272febb41a292c6bf632baaedbc, State: Initialized, Role: FOLLOWER
I20260618 22:33:33.727521 26472 consensus_queue.cc:260] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } }
I20260618 22:33:33.728502 26473 ts_tablet_manager.cc:1428] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: Time spent starting tablet: real 0.019s user 0.012s sys 0.006s
I20260618 22:33:33.730314 26298 heartbeater.cc:503] Master 127.18.150.190:35785 was elected leader, sending a full tablet report...
I20260618 22:33:33.730846 26472 ts_tablet_manager.cc:1428] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: Time spent starting tablet: real 0.019s user 0.014s sys 0.000s
W20260618 22:33:33.730921 26300 tablet.cc:2369] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:33:33.809930 26441 tablet.cc:2369] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:33:33.916442 26153 tablet.cc:2369] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:33:34.036650 26478 raft_consensus.cc:482] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:33:34.037009 26478 raft_consensus.cc:504] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } }
I20260618 22:33:34.038745 26478 leader_election.cc:288] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7632b2891444502858ae543d072076e (127.18.150.130:37773), d3b54272febb41a292c6bf632baaedbc (127.18.150.129:43405)
I20260618 22:33:34.046171 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:34.039123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:37711 (local address 127.18.150.130:37773)
0618 22:33:34.039326 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:34.039332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:34.039539 (+ 207us) server_negotiation.cc:407] Connection header received
0618 22:33:34.039670 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:34.039674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:34.039739 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:34.039821 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:34.040189 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.041203 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.043867 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.045101 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.045253 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:34.045327 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:34.045733 (+ 406us) server_negotiation.cc:299] Negotiation successful
0618 22:33:34.045885 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:33:34.046171 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:34.039224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:48673 (local address 127.18.150.129:43405)
0618 22:33:34.039428 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:34.039433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:34.039448 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:33:34.039504 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:34.039509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:34.039578 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:34.039678 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:34.040083 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.041181 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.043889 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.045099 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.045253 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:34.045327 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:34.045733 (+ 406us) server_negotiation.cc:299] Negotiation successful
0618 22:33:34.045885 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:33:34.046252 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:34.038968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43405 (local address 127.18.150.131:48673)
0618 22:33:34.039144 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:34.039164 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:34.039277 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:34.039706 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:34.039710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:34.039729 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:34.039919 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:34.039930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.041342 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.041347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:34.043692 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:34.043709 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.045279 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.045285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:34.045526 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:34.045610 (+ 84us) client_negotiation.cc:769] Sending connection context
0618 22:33:34.045708 (+ 98us) client_negotiation.cc:240] Negotiation successful
0618 22:33:34.045901 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":190}
I20260618 22:33:34.046257 26445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:34.038968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:37773 (local address 127.18.150.131:37711)
0618 22:33:34.039454 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:34.039468 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:34.039559 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:34.039852 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:34.039857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:34.039876 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:34.040032 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:34.040041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.041336 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.041342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:34.043691 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:34.043709 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.045279 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.045285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:34.045542 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:34.045616 (+ 74us) client_negotiation.cc:769] Sending connection context
0618 22:33:34.045708 (+ 92us) client_negotiation.cc:240] Negotiation successful
0618 22:33:34.045907 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":587}
I20260618 22:33:34.047313 26102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6280d444ca4d6eb68c4b343d70a8ad" candidate_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3b54272febb41a292c6bf632baaedbc" is_pre_election: true
I20260618 22:33:34.047454 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6280d444ca4d6eb68c4b343d70a8ad" candidate_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7632b2891444502858ae543d072076e" is_pre_election: true
I20260618 22:33:34.047892 26102 raft_consensus.cc:2454] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6a71f206db54fe4afde3cc0415e0ba6 in term 0.
I20260618 22:33:34.047892 26252 raft_consensus.cc:2454] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6a71f206db54fe4afde3cc0415e0ba6 in term 0.
I20260618 22:33:34.048527 26328 leader_election.cc:302] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3b54272febb41a292c6bf632baaedbc, f6a71f206db54fe4afde3cc0415e0ba6; no voters:
I20260618 22:33:34.048875 26478 raft_consensus.cc:2781] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:33:34.049015 26478 raft_consensus.cc:482] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:33:34.049120 26478 raft_consensus.cc:3037] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:34.051242 26478 raft_consensus.cc:504] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } }
I20260618 22:33:34.051788 26478 leader_election.cc:288] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [CANDIDATE]: Term 1 election: Requested vote from peers d7632b2891444502858ae543d072076e (127.18.150.130:37773), d3b54272febb41a292c6bf632baaedbc (127.18.150.129:43405)
I20260618 22:33:34.052189 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6280d444ca4d6eb68c4b343d70a8ad" candidate_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7632b2891444502858ae543d072076e"
I20260618 22:33:34.052228 26102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6280d444ca4d6eb68c4b343d70a8ad" candidate_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3b54272febb41a292c6bf632baaedbc"
I20260618 22:33:34.052367 26252 raft_consensus.cc:3037] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:34.052399 26102 raft_consensus.cc:3037] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:34.054507 26252 raft_consensus.cc:2454] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6a71f206db54fe4afde3cc0415e0ba6 in term 1.
I20260618 22:33:34.054539 26102 raft_consensus.cc:2454] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6a71f206db54fe4afde3cc0415e0ba6 in term 1.
I20260618 22:33:34.054878 26326 leader_election.cc:302] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7632b2891444502858ae543d072076e, f6a71f206db54fe4afde3cc0415e0ba6; no voters:
I20260618 22:33:34.055152 26478 raft_consensus.cc:2781] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:34.055537 26478 raft_consensus.cc:686] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 1 LEADER]: Becoming Leader. State: Replica: f6a71f206db54fe4afde3cc0415e0ba6, State: Running, Role: LEADER
I20260618 22:33:34.056119 26478 consensus_queue.cc:237] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } }
I20260618 22:33:34.061050 25182 catalog_manager.cc:5495] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6a71f206db54fe4afde3cc0415e0ba6 (127.18.150.131). New cstate: current_term: 1 leader_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260618 22:33:34.137035 26468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:34.132054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46990 (local address 127.18.150.131:38539)
0618 22:33:34.132217 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:34.132223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:34.132240 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:33:34.132379 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:34.132382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:34.132457 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:34.132578 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:33:34.132857 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.133517 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.134967 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.135457 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.135555 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:34.136006 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:33:34.136536 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:33:34.136681 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:34.136739 (+ 58us) server_negotiation.cc:299] Negotiation successful
0618 22:33:34.136824 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:33:34.144789 26252 raft_consensus.cc:1261] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e [term 1 FOLLOWER]: Refusing update from remote peer f6a71f206db54fe4afde3cc0415e0ba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:34.144790 26102 raft_consensus.cc:1261] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc [term 1 FOLLOWER]: Refusing update from remote peer f6a71f206db54fe4afde3cc0415e0ba6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:34.145754 26481 consensus_queue.cc:1035] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:34.146099 26478 consensus_queue.cc:1035] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:34.158402 26484 mvcc.cc:205] Tried to move back new op lower bound from 7298342969927376896 to 7298342969577558016. Current Snapshot: MvccSnapshot[applied={T|T < 7298342969927376896}]
I20260618 22:33:34.176697 25182 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:34.176916 25182 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:34.184049 25182 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b9615ea036b408b8b1da7b8b15b4263: Sending DeleteTablet for 3 replicas of tablet ba6280d444ca4d6eb68c4b343d70a8ad
I20260618 22:33:34.185082 26232 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6280d444ca4d6eb68c4b343d70a8ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:34 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:32810
I20260618 22:33:34.185483 26373 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6280d444ca4d6eb68c4b343d70a8ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:34 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:46984
I20260618 22:33:34.185495 26082 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6280d444ca4d6eb68c4b343d70a8ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:34 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:43062
I20260618 22:33:34.195389 26495 tablet_replica.cc:330] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: stopping tablet replica
I20260618 22:33:34.195916 26495 raft_consensus.cc:2229] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:33:34.196623 26495 raft_consensus.cc:2258] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:34.198343 26496 tablet_replica.cc:330] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: stopping tablet replica
I20260618 22:33:34.198907 26496 raft_consensus.cc:2229] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:34.199440 26496 raft_consensus.cc:2258] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:34.201802 26495 ts_tablet_manager.cc:1905] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:34.208392 26494 tablet_replica.cc:330] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: stopping tablet replica
I20260618 22:33:34.208814 26494 raft_consensus.cc:2229] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:34.209263 26494 raft_consensus.cc:2258] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:34.210898 26496 ts_tablet_manager.cc:1905] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:34.211207 26494 ts_tablet_manager.cc:1905] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:34.217625 26494 ts_tablet_manager.cc:1918] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:34.217621 26496 ts_tablet_manager.cc:1918] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:34.217792 26496 log.cc:1196] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/ba6280d444ca4d6eb68c4b343d70a8ad
I20260618 22:33:34.217792 26494 log.cc:1196] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/ba6280d444ca4d6eb68c4b343d70a8ad
I20260618 22:33:34.218348 26494 ts_tablet_manager.cc:1939] T ba6280d444ca4d6eb68c4b343d70a8ad P d7632b2891444502858ae543d072076e: Deleting consensus metadata
I20260618 22:33:34.218344 26496 ts_tablet_manager.cc:1939] T ba6280d444ca4d6eb68c4b343d70a8ad P d3b54272febb41a292c6bf632baaedbc: Deleting consensus metadata
I20260618 22:33:34.220093 25168 catalog_manager.cc:4841] TS d7632b2891444502858ae543d072076e (127.18.150.130:37773): tablet ba6280d444ca4d6eb68c4b343d70a8ad (table test-table [id=9bd875589d884646a5acea7c4da3afcb]) successfully deleted
I20260618 22:33:34.220103 25170 catalog_manager.cc:4841] TS d3b54272febb41a292c6bf632baaedbc (127.18.150.129:43405): tablet ba6280d444ca4d6eb68c4b343d70a8ad (table test-table [id=9bd875589d884646a5acea7c4da3afcb]) successfully deleted
I20260618 22:33:34.221709 26495 ts_tablet_manager.cc:1918] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:34.221879 26495 log.cc:1196] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/ba6280d444ca4d6eb68c4b343d70a8ad
I20260618 22:33:34.222400 26495 ts_tablet_manager.cc:1939] T ba6280d444ca4d6eb68c4b343d70a8ad P f6a71f206db54fe4afde3cc0415e0ba6: Deleting consensus metadata
I20260618 22:33:34.223685 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 25152
I20260618 22:33:34.224205 25171 catalog_manager.cc:4841] TS f6a71f206db54fe4afde3cc0415e0ba6 (127.18.150.131:38539): tablet ba6280d444ca4d6eb68c4b343d70a8ad (table test-table [id=9bd875589d884646a5acea7c4da3afcb]) successfully deleted
I20260618 22:33:34.225292 25214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:34.239168 26498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:34.188937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53304 (local address 127.18.150.189:37051)
0618 22:33:34.216837 (+ 27900us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:34.216845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:34.217657 (+ 812us) server_negotiation.cc:407] Connection header received
0618 22:33:34.217978 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:34.217986 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:34.218073 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:34.218181 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:34.220957 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.222293 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.226528 (+ 4235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.227183 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.227309 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:34.230936 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:34.230944 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:34.230949 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:34.230995 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:34.235232 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:34.236234 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:34.236240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:34.236244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:34.236403 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:34.236814 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:34.236819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:34.236822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:34.237166 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:34.237394 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:34.238562 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0618 22:33:34.238795 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27748,"thread_start_us":27930,"threads_started":1}
I20260618 22:33:34.255475 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:34.223044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49712 (local address 127.18.150.188:34705)
0618 22:33:34.239656 (+ 16612us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:34.239662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:34.240770 (+ 1108us) server_negotiation.cc:407] Connection header received
0618 22:33:34.240916 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:34.240923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:34.241047 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:34.241172 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:34.242968 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.243733 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.245020 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.245463 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.245579 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:34.248157 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:34.248165 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:34.248170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:34.248211 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:34.251647 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:34.252498 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:34.252504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:34.252508 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:34.252649 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:34.253049 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:34.253054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:34.253058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:34.253428 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:34.254708 (+ 1280us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:34.254828 (+ 120us) server_negotiation.cc:299] Negotiation successful
0618 22:33:34.255085 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16476,"thread_start_us":16610,"threads_started":1}
I20260618 22:33:34.379954 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 25152
I20260618 22:33:34.439613 26498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:34.409398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.18.150.189:37051)
0618 22:33:34.410035 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:34.410043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:34.414973 (+ 4930us) server_negotiation.cc:407] Connection header received
0618 22:33:34.415337 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:34.415350 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:34.415468 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:34.415570 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:34.427061 (+ 11491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.427875 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.429797 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.430470 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.430604 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:34.433395 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:34.433402 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:34.433407 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:34.433450 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:34.436943 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:34.437534 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:34.437539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:34.437543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:34.437686 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:34.438093 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:34.438097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:34.438100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:34.438443 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:34.438688 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:34.439075 (+ 387us) server_negotiation.cc:299] Negotiation successful
0618 22:33:34.439296 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394}
I20260618 22:33:34.442631 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:34.408806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49724 (local address 127.18.150.188:34705)
0618 22:33:34.409041 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:34.409047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:34.426734 (+ 17687us) server_negotiation.cc:407] Connection header received
0618 22:33:34.428100 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:34.428107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:34.428197 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:34.428267 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:34.430448 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.431398 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.433295 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:34.433745 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:34.433858 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:34.436337 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:34.436343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:34.436347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:34.436380 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:34.439675 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:34.440627 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:34.440633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:34.440637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:34.440797 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:34.441242 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:34.441246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:34.441249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:34.441513 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:34.441686 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:34.442093 (+ 407us) server_negotiation.cc:299] Negotiation successful
0618 22:33:34.442336 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260618 22:33:35.168193 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.167314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.131:60093)
0618 22:33:35.167838 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.167987 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":260,"threads_started":1}
I20260618 22:33:35.168193 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.167366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.130:59995)
0618 22:33:35.167870 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.167987 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":299,"threads_started":1}
W20260618 22:33:35.168630 26298 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:35785 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
W20260618 22:33:35.168632 26439 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:35785 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
I20260618 22:33:35.174018 26510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.172042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.129:36487)
0618 22:33:35.173720 (+ 1678us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.173828 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1573,"thread_start_us":1473,"threads_started":1}
W20260618 22:33:35.174384 26151 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:35785 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
I20260618 22:33:35.683905 26511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b9615ea036b408b8b1da7b8b15b4263)
I20260618 22:33:35.684134 26511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } }
I20260618 22:33:35.685041 26512 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2b9615ea036b408b8b1da7b8b15b4263)
I20260618 22:33:35.685257 26512 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } }
I20260618 22:33:35.685598 26511 leader_election.cc:288] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b9615ea036b408b8b1da7b8b15b4263 (127.18.150.190:35785), 863af7c92037482dbd2d470608c0a984 (127.18.150.189:37051)
I20260618 22:33:35.686801 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.685701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:32822)
0618 22:33:35.686317 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.686512 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":405,"threads_started":1}
W20260618 22:33:35.687040 25956 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
I20260618 22:33:35.692941 26512 leader_election.cc:288] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b9615ea036b408b8b1da7b8b15b4263 (127.18.150.190:35785), 0857f29096284b568f1731395c320ab1 (127.18.150.188:34705)
I20260618 22:33:35.694667 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.685832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:37051 (local address 127.0.0.1:53328)
0618 22:33:35.686985 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.687005 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:35.687168 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:35.687931 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:35.687936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:35.687965 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:35.688223 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.688234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.689972 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.689978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.692037 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.692052 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.693823 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.693830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.694039 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.694079 (+ 40us) client_negotiation.cc:769] Sending connection context
0618 22:33:35.694227 (+ 148us) client_negotiation.cc:240] Negotiation successful
0618 22:33:35.694392 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"threads_started":1}
I20260618 22:33:35.695526 26515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.685966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.18.150.189:37051)
0618 22:33:35.687624 (+ 1658us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:35.687629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:35.687646 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:33:35.687709 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:35.687714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:35.687779 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:35.688353 (+ 574us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:35.688575 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.689777 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.692248 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.693569 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.694905 (+ 1336us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.694995 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:35.695048 (+ 53us) server_negotiation.cc:299] Negotiation successful
0618 22:33:35.695191 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1527,"thread_start_us":1416,"threads_started":1}
I20260618 22:33:35.697391 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.687687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:32828)
0618 22:33:35.696975 (+ 9288us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.697154 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9166,"thread_start_us":9041,"threads_started":1}
I20260618 22:33:35.697903 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.697305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:32840)
0618 22:33:35.697532 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.697703 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260618 22:33:35.697780 25792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0857f29096284b568f1731395c320ab1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "863af7c92037482dbd2d470608c0a984" is_pre_election: true
W20260618 22:33:35.698108 25956 leader_election.cc:334] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b9615ea036b408b8b1da7b8b15b4263 (127.18.150.190:35785): Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
I20260618 22:33:35.698175 25792 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0857f29096284b568f1731395c320ab1 in term 1.
W20260618 22:33:35.698408 25768 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
I20260618 22:33:35.699265 25957 leader_election.cc:302] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0857f29096284b568f1731395c320ab1, 863af7c92037482dbd2d470608c0a984; no voters: 2b9615ea036b408b8b1da7b8b15b4263
I20260618 22:33:35.699618 26511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 22:33:35.699779 26511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2b9615ea036b408b8b1da7b8b15b4263)
I20260618 22:33:35.699859 26511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:33:35.699929 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.699527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:32856)
0618 22:33:35.699723 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.699805 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260618 22:33:35.700099 25768 leader_election.cc:334] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b9615ea036b408b8b1da7b8b15b4263 (127.18.150.190:35785): Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
I20260618 22:33:35.702389 26511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } }
I20260618 22:33:35.703054 26511 leader_election.cc:288] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [CANDIDATE]: Term 2 election: Requested vote from peers 2b9615ea036b408b8b1da7b8b15b4263 (127.18.150.190:35785), 863af7c92037482dbd2d470608c0a984 (127.18.150.189:37051)
I20260618 22:33:35.703172 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.686896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:34705 (local address 127.0.0.1:49726)
0618 22:33:35.696706 (+ 9810us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.696725 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:35.696854 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:35.697652 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:35.697656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:35.697681 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:35.698018 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.698031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.699531 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.699536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.701089 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.701100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.702570 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.702574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.702713 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.702737 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:33:35.702813 (+ 76us) client_negotiation.cc:240] Negotiation successful
0618 22:33:35.702896 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9676,"thread_start_us":9546,"threads_started":1}
I20260618 22:33:35.703480 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.696030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.18.150.188:34705)
0618 22:33:35.697280 (+ 1250us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:35.697286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:35.697309 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:33:35.697365 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:35.697370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:35.697444 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:35.697542 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:35.698212 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.699392 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.701245 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.702434 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.702598 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.702666 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:35.702852 (+ 186us) server_negotiation.cc:299] Negotiation successful
0618 22:33:35.702977 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1126,"thread_start_us":1013,"threads_started":1}
I20260618 22:33:35.703542 25792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0857f29096284b568f1731395c320ab1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "863af7c92037482dbd2d470608c0a984"
I20260618 22:33:35.703624 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.703169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:32860)
0618 22:33:35.703397 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.703488 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260618 22:33:35.703742 25792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:33:35.704236 25980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "863af7c92037482dbd2d470608c0a984" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0857f29096284b568f1731395c320ab1" is_pre_election: true
I20260618 22:33:35.704504 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.704163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:32862)
0618 22:33:35.704290 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.704360 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260618 22:33:35.704586 25980 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 863af7c92037482dbd2d470608c0a984 in current term 2: Already voted for candidate 0857f29096284b568f1731395c320ab1 in this term.
W20260618 22:33:35.704676 25956 leader_election.cc:334] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2b9615ea036b408b8b1da7b8b15b4263 (127.18.150.190:35785): Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
I20260618 22:33:35.705137 25766 leader_election.cc:302] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 863af7c92037482dbd2d470608c0a984; no voters: 0857f29096284b568f1731395c320ab1, 2b9615ea036b408b8b1da7b8b15b4263
I20260618 22:33:35.706086 25792 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0857f29096284b568f1731395c320ab1 in term 2.
I20260618 22:33:35.706238 26512 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:33:35.706465 25957 leader_election.cc:302] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0857f29096284b568f1731395c320ab1, 863af7c92037482dbd2d470608c0a984; no voters: 2b9615ea036b408b8b1da7b8b15b4263
I20260618 22:33:35.706694 26511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:33:35.707109 26511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [term 2 LEADER]: Becoming Leader. State: Replica: 0857f29096284b568f1731395c320ab1, State: Running, Role: LEADER
I20260618 22:33:35.707453 26511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } }
I20260618 22:33:35.709684 26521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0857f29096284b568f1731395c320ab1. Latest consensus state: current_term: 2 leader_uuid: "0857f29096284b568f1731395c320ab1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:35.709960 26521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:35.710500 26523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:35.713896 26523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:35.714654 26523 catalog_manager.cc:1259] Loaded cluster ID: 6965aa89c6e8419c865b29d8b8ae8be6
I20260618 22:33:35.714735 26523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:35.715777 26523 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:35.716485 26523 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: Loaded TSK: 0
I20260618 22:33:35.717257 26523 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:33:35.778393 25969 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:33:35.785157 25969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:33:35.789019 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.788672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:32900)
0618 22:33:35.788823 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.788895 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260618 22:33:35.789191 25792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [term 2 FOLLOWER]: Refusing update from remote peer 0857f29096284b568f1731395c320ab1: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260618 22:33:35.789896 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.789569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:32908)
0618 22:33:35.789701 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.789772 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260618 22:33:35.789980 26521 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260618 22:33:35.790258 25956 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 -> Peer 2b9615ea036b408b8b1da7b8b15b4263 (127.18.150.190:35785): Couldn't send request to peer 2b9615ea036b408b8b1da7b8b15b4263. Status: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260618 22:33:35.792621 26512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0857f29096284b568f1731395c320ab1. Latest consensus state: current_term: 2 leader_uuid: "0857f29096284b568f1731395c320ab1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:35.792905 26512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:33:35.794323 26511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "0857f29096284b568f1731395c320ab1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:35.794608 26511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:35.794984 26525 mvcc.cc:205] Tried to move back new op lower bound from 7298342976666415104 to 7298342976341135360. Current Snapshot: MvccSnapshot[applied={T|T < 7298342976666415104}]
I20260618 22:33:35.795630 26512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "0857f29096284b568f1731395c320ab1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "2b9615ea036b408b8b1da7b8b15b4263" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35785 } } peers { permanent_uuid: "863af7c92037482dbd2d470608c0a984" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 37051 } attrs { promote: false } } peers { permanent_uuid: "0857f29096284b568f1731395c320ab1" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 34705 } attrs { promote: false } } }
I20260618 22:33:35.795909 26512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863af7c92037482dbd2d470608c0a984 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:33:35.808872 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.802688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:38539 (local address 127.0.0.1:47004)
0618 22:33:35.802894 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.802912 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:35.803015 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:35.803835 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:35.803840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:35.803866 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:35.804140 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.804156 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.805412 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.805417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.806880 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.806894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.808152 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.808159 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.808352 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.808389 (+ 37us) client_negotiation.cc:769] Sending connection context
0618 22:33:35.808456 (+ 67us) client_negotiation.cc:240] Negotiation successful
0618 22:33:35.808579 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260618 22:33:35.809749 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.802801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47004 (local address 127.18.150.131:38539)
0618 22:33:35.803505 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:35.803511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:35.803529 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:33:35.803598 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:35.803603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:35.803681 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:35.803774 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:35.804269 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.805268 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.807038 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.807910 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.809372 (+ 1462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.809421 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:35.809456 (+ 35us) server_negotiation.cc:299] Negotiation successful
0618 22:33:35.809548 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":1042,"threads_started":1}
I20260618 22:33:35.810220 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.803420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43064 (local address 127.18.150.129:43405)
0618 22:33:35.803826 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:35.803831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:35.803847 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:33:35.803905 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:35.803910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:35.803983 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:35.804080 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:35.804540 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.805526 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.807667 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.808866 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.809133 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.809212 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:35.809816 (+ 604us) server_negotiation.cc:299] Negotiation successful
0618 22:33:35.809982 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":220,"threads_started":1}
I20260618 22:33:35.810492 26373 tablet_service.cc:1467] Processing CreateTablet for tablet 5c86a98cebc2484696b37c911b3cc050 (DEFAULT_TABLE table=test-table [id=a816dfc9f5994a6d92fe61bdc32496d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:35.810575 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.802967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43405 (local address 127.0.0.1:43064)
0618 22:33:35.803182 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.803196 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:35.803261 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:35.804122 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:35.804128 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:35.804174 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:35.804394 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.804404 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.805654 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.805660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.807506 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.807520 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.809444 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.809451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.809645 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.809682 (+ 37us) client_negotiation.cc:769] Sending connection context
0618 22:33:35.809770 (+ 88us) client_negotiation.cc:240] Negotiation successful
0618 22:33:35.810144 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260618 22:33:35.810899 26373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c86a98cebc2484696b37c911b3cc050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:35.811398 26082 tablet_service.cc:1467] Processing CreateTablet for tablet 5c86a98cebc2484696b37c911b3cc050 (DEFAULT_TABLE table=test-table [id=a816dfc9f5994a6d92fe61bdc32496d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:35.811820 26082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c86a98cebc2484696b37c911b3cc050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:35.816794 26534 tablet_bootstrap.cc:492] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6: Bootstrap starting.
I20260618 22:33:35.819113 26535 tablet_bootstrap.cc:492] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc: Bootstrap starting.
I20260618 22:33:35.819322 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.803387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:37773 (local address 127.0.0.1:32824)
0618 22:33:35.809086 (+ 5699us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.809102 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:35.809196 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:35.814141 (+ 4945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:35.814147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:35.814172 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:35.814325 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.814335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.816144 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.816149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.817375 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.817387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.818554 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.818560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.818747 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.818782 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:33:35.818868 (+ 86us) client_negotiation.cc:240] Negotiation successful
0618 22:33:35.819003 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5616,"thread_start_us":11306,"threads_started":1}
I20260618 22:33:35.819900 26534 tablet_bootstrap.cc:654] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:35.820261 26533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.803729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32824 (local address 127.18.150.130:37773)
0618 22:33:35.813804 (+ 10075us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:35.813810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:35.813829 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:33:35.813893 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:35.813899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:35.813992 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:35.814451 (+ 459us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:35.814999 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.815983 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.817534 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.818418 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.818560 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.818652 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:35.819777 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0618 22:33:35.819932 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9955,"thread_start_us":9854,"threads_started":1}
I20260618 22:33:35.820761 26535 tablet_bootstrap.cc:654] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:35.820971 26232 tablet_service.cc:1467] Processing CreateTablet for tablet 5c86a98cebc2484696b37c911b3cc050 (DEFAULT_TABLE table=test-table [id=a816dfc9f5994a6d92fe61bdc32496d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:35.821393 26232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c86a98cebc2484696b37c911b3cc050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:35.822328 26534 tablet_bootstrap.cc:492] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6: No bootstrap required, opened a new log
I20260618 22:33:35.822465 26534 ts_tablet_manager.cc:1397] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260618 22:33:35.823226 26534 raft_consensus.cc:348] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } }
I20260618 22:33:35.823450 26534 raft_consensus.cc:374] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:35.823544 26534 raft_consensus.cc:729] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a71f206db54fe4afde3cc0415e0ba6, State: Initialized, Role: FOLLOWER
I20260618 22:33:35.823580 26535 tablet_bootstrap.cc:492] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc: No bootstrap required, opened a new log
I20260618 22:33:35.823802 26535 ts_tablet_manager.cc:1397] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:33:35.823760 26534 consensus_queue.cc:260] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } }
I20260618 22:33:35.824572 26534 ts_tablet_manager.cc:1428] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:33:35.824745 26535 raft_consensus.cc:348] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } }
I20260618 22:33:35.824982 26535 raft_consensus.cc:374] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:35.825080 26535 raft_consensus.cc:729] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3b54272febb41a292c6bf632baaedbc, State: Initialized, Role: FOLLOWER
I20260618 22:33:35.825320 26535 consensus_queue.cc:260] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } }
I20260618 22:33:35.825975 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.824928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.131:33991)
0618 22:33:35.825468 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.825711 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"threads_started":1}
I20260618 22:33:35.827380 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.826991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.131:37079)
0618 22:33:35.827174 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.827254 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260618 22:33:35.827957 26438 heartbeater.cc:503] Master 127.18.150.188:34705 was elected leader, sending a full tablet report...
I20260618 22:33:35.830449 26535 ts_tablet_manager.cc:1428] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260618 22:33:35.832203 26150 heartbeater.cc:503] Master 127.18.150.188:34705 was elected leader, sending a full tablet report...
I20260618 22:33:35.835330 26541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.830613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.129:33489)
0618 22:33:35.834852 (+ 4239us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.835043 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3715,"thread_start_us":3666,"threads_started":1}
I20260618 22:33:35.836239 26542 tablet_bootstrap.cc:492] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e: Bootstrap starting.
I20260618 22:33:35.836459 26541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.835962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.129:59231)
0618 22:33:35.836262 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.836341 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260618 22:33:35.837672 26542 tablet_bootstrap.cc:654] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:35.839402 26542 tablet_bootstrap.cc:492] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e: No bootstrap required, opened a new log
I20260618 22:33:35.839515 26542 ts_tablet_manager.cc:1397] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260618 22:33:35.840096 26542 raft_consensus.cc:348] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } }
I20260618 22:33:35.840287 26542 raft_consensus.cc:374] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:35.840373 26542 raft_consensus.cc:729] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7632b2891444502858ae543d072076e, State: Initialized, Role: FOLLOWER
I20260618 22:33:35.840590 26542 consensus_queue.cc:260] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } }
I20260618 22:33:35.841485 26542 ts_tablet_manager.cc:1428] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:33:35.843437 26545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.842481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.130:44553)
0618 22:33:35.843002 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.843156 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":249,"threads_started":1}
I20260618 22:33:35.843873 26297 heartbeater.cc:503] Master 127.18.150.188:34705 was elected leader, sending a full tablet report...
I20260618 22:33:35.845047 26545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.844510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.130:54651)
0618 22:33:35.844800 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.844885 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"threads_started":1}
I20260618 22:33:35.875094 26540 raft_consensus.cc:482] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:33:35.875293 26540 raft_consensus.cc:504] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } }
I20260618 22:33:35.876700 26540 leader_election.cc:288] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6a71f206db54fe4afde3cc0415e0ba6 (127.18.150.131:38539), d7632b2891444502858ae543d072076e (127.18.150.130:37773)
I20260618 22:33:35.883399 26541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.876926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:38539 (local address 127.18.150.129:60019)
0618 22:33:35.877166 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.877193 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:35.877347 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:35.877726 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:35.877732 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:35.877755 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:35.878016 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.878026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.879282 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.879285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.881414 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.881433 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.882725 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.882731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.882900 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.882935 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:33:35.883025 (+ 90us) client_negotiation.cc:240] Negotiation successful
0618 22:33:35.883176 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260618 22:33:35.883419 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.877076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:60019 (local address 127.18.150.131:38539)
0618 22:33:35.877469 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:35.877474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:35.877493 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:33:35.877560 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:35.877564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:35.877620 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:35.877695 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:35.878163 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.879145 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.881627 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.882529 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.882674 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.882727 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:35.883052 (+ 325us) server_negotiation.cc:299] Negotiation successful
0618 22:33:35.883185 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260618 22:33:35.883774 26533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.877073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:58141 (local address 127.18.150.130:37773)
0618 22:33:35.877321 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:35.877327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:35.877624 (+ 297us) server_negotiation.cc:407] Connection header received
0618 22:33:35.877824 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:35.877830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:35.877886 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:35.877982 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:35.878466 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.879409 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.881599 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.882859 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.883011 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.883076 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:35.883409 (+ 333us) server_negotiation.cc:299] Negotiation successful
0618 22:33:35.883531 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260618 22:33:35.884128 26393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c86a98cebc2484696b37c911b3cc050" candidate_uuid: "d3b54272febb41a292c6bf632baaedbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" is_pre_election: true
I20260618 22:33:35.884461 26393 raft_consensus.cc:2454] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3b54272febb41a292c6bf632baaedbc in term 0.
I20260618 22:33:35.883843 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.876954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:37773 (local address 127.18.150.129:58141)
0618 22:33:35.877534 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.877546 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:35.877654 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:35.878018 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:35.878022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:35.878045 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:35.878332 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.878341 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.879513 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.879518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.881445 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:35.881454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.883056 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.883063 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:35.883239 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.883280 (+ 41us) client_negotiation.cc:769] Sending connection context
0618 22:33:35.883365 (+ 85us) client_negotiation.cc:240] Negotiation successful
0618 22:33:35.883488 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":1,"thread_start_us":332,"threads_started":1}
I20260618 22:33:35.885040 26038 leader_election.cc:302] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3b54272febb41a292c6bf632baaedbc, f6a71f206db54fe4afde3cc0415e0ba6; no voters:
I20260618 22:33:35.885290 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c86a98cebc2484696b37c911b3cc050" candidate_uuid: "d3b54272febb41a292c6bf632baaedbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7632b2891444502858ae543d072076e" is_pre_election: true
I20260618 22:33:35.885460 26540 raft_consensus.cc:2781] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:33:35.885532 26252 raft_consensus.cc:2454] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3b54272febb41a292c6bf632baaedbc in term 0.
I20260618 22:33:35.885568 26540 raft_consensus.cc:482] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:33:35.885651 26540 raft_consensus.cc:3037] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:35.887740 26540 raft_consensus.cc:504] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } }
I20260618 22:33:35.888273 26540 leader_election.cc:288] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [CANDIDATE]: Term 1 election: Requested vote from peers f6a71f206db54fe4afde3cc0415e0ba6 (127.18.150.131:38539), d7632b2891444502858ae543d072076e (127.18.150.130:37773)
I20260618 22:33:35.888670 26393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c86a98cebc2484696b37c911b3cc050" candidate_uuid: "d3b54272febb41a292c6bf632baaedbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a71f206db54fe4afde3cc0415e0ba6"
I20260618 22:33:35.888698 26252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c86a98cebc2484696b37c911b3cc050" candidate_uuid: "d3b54272febb41a292c6bf632baaedbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7632b2891444502858ae543d072076e"
I20260618 22:33:35.888841 26393 raft_consensus.cc:3037] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:35.888860 26252 raft_consensus.cc:3037] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:35.891106 26393 raft_consensus.cc:2454] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3b54272febb41a292c6bf632baaedbc in term 1.
I20260618 22:33:35.891139 26252 raft_consensus.cc:2454] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3b54272febb41a292c6bf632baaedbc in term 1.
I20260618 22:33:35.891445 26038 leader_election.cc:302] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3b54272febb41a292c6bf632baaedbc, f6a71f206db54fe4afde3cc0415e0ba6; no voters:
I20260618 22:33:35.891690 26540 raft_consensus.cc:2781] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:35.892061 26540 raft_consensus.cc:686] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 1 LEADER]: Becoming Leader. State: Replica: d3b54272febb41a292c6bf632baaedbc, State: Running, Role: LEADER
I20260618 22:33:35.892433 26540 consensus_queue.cc:237] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } } peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } }
I20260618 22:33:35.892995 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.892583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.129:58129)
0618 22:33:35.892735 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.892855 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260618 22:33:35.895954 25969 catalog_manager.cc:5495] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc reported cstate change: term changed from 0 to 1, leader changed from <none> to d3b54272febb41a292c6bf632baaedbc (127.18.150.129). New cstate: current_term: 1 leader_uuid: "d3b54272febb41a292c6bf632baaedbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3b54272febb41a292c6bf632baaedbc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 } health_report { overall_health: UNKNOWN } } }
I20260618 22:33:35.921005 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.915618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.18.150.129:43405)
0618 22:33:35.915816 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:35.915822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:35.915840 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:33:35.916043 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:35.916047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:35.916120 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:35.916242 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:33:35.916579 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.917424 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.918792 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.919237 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.919338 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.919691 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:33:35.920349 (+ 658us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:33:35.920488 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:35.920585 (+ 97us) server_negotiation.cc:299] Negotiation successful
0618 22:33:35.920669 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:33:35.924849 26252 raft_consensus.cc:1261] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e [term 1 FOLLOWER]: Refusing update from remote peer d3b54272febb41a292c6bf632baaedbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:35.925136 26393 raft_consensus.cc:1261] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6 [term 1 FOLLOWER]: Refusing update from remote peer d3b54272febb41a292c6bf632baaedbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:35.925491 26548 consensus_queue.cc:1035] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7632b2891444502858ae543d072076e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 37773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:35.925836 26540 consensus_queue.cc:1035] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6a71f206db54fe4afde3cc0415e0ba6" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:35.928532 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.927966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.131:35239)
0618 22:33:35.928206 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.928322 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260618 22:33:35.928747 26546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.928135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.130:44359)
0618 22:33:35.928300 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.928495 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260618 22:33:35.933990 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.933447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.131:54675)
0618 22:33:35.933675 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.933782 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260618 22:33:35.934716 26551 mvcc.cc:205] Tried to move back new op lower bound from 7298342977221902336 to 7298342977099436032. Current Snapshot: MvccSnapshot[applied={T|T < 7298342977221902336}]
I20260618 22:33:35.936806 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.936350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.129:54475)
0618 22:33:35.936563 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.936647 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260618 22:33:35.937033 26552 mvcc.cc:205] Tried to move back new op lower bound from 7298342977221902336 to 7298342977099436032. Current Snapshot: MvccSnapshot[applied={T|T < 7298342977221902336}]
I20260618 22:33:35.937642 26546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.936716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.130:40211)
0618 22:33:35.937390 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.937487 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":559}
I20260618 22:33:35.937821 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.937416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.18.150.129:55691)
0618 22:33:35.937607 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:35.937695 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260618 22:33:35.952610 25970 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:35.952924 25970 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:35.961849 25970 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0857f29096284b568f1731395c320ab1: Sending DeleteTablet for 3 replicas of tablet 5c86a98cebc2484696b37c911b3cc050
I20260618 22:33:35.963445 26232 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c86a98cebc2484696b37c911b3cc050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:35 UTC) from {username='oryx', principal='oryx/127.18.150.188@KRBTEST.COM'} at 127.0.0.1:32824
I20260618 22:33:35.963907 26373 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c86a98cebc2484696b37c911b3cc050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:35 UTC) from {username='oryx', principal='oryx/127.18.150.188@KRBTEST.COM'} at 127.0.0.1:47004
I20260618 22:33:35.965144 26082 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c86a98cebc2484696b37c911b3cc050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:35 UTC) from {username='oryx', principal='oryx/127.18.150.188@KRBTEST.COM'} at 127.0.0.1:43064
I20260618 22:33:35.966234 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 25951
I20260618 22:33:35.984639 26562 tablet_replica.cc:330] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6: stopping tablet replica
I20260618 22:33:35.984838 26562 raft_consensus.cc:2229] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:35.985155 26562 raft_consensus.cc:2258] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:35.986394 26562 ts_tablet_manager.cc:1905] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:35.988610 26561 tablet_replica.cc:330] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e: stopping tablet replica
I20260618 22:33:35.989392 26561 raft_consensus.cc:2229] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:35.990111 26561 raft_consensus.cc:2258] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:35.990473 26563 tablet_replica.cc:330] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc: stopping tablet replica
I20260618 22:33:35.990667 26563 raft_consensus.cc:2229] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:33:35.991297 26563 raft_consensus.cc:2258] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:35.995067 26562 ts_tablet_manager.cc:1918] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:35.995034 26561 ts_tablet_manager.cc:1905] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:35.995203 26562 log.cc:1196] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/5c86a98cebc2484696b37c911b3cc050
I20260618 22:33:35.995582 26562 ts_tablet_manager.cc:1939] T 5c86a98cebc2484696b37c911b3cc050 P f6a71f206db54fe4afde3cc0415e0ba6: Deleting consensus metadata
I20260618 22:33:35.996628 25957 catalog_manager.cc:4841] TS f6a71f206db54fe4afde3cc0415e0ba6 (127.18.150.131:38539): tablet 5c86a98cebc2484696b37c911b3cc050 (table test-table [id=a816dfc9f5994a6d92fe61bdc32496d7]) successfully deleted
I20260618 22:33:35.998934 26563 ts_tablet_manager.cc:1905] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:36.002168 26561 ts_tablet_manager.cc:1918] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:36.002295 26561 log.cc:1196] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/5c86a98cebc2484696b37c911b3cc050
I20260618 22:33:36.002699 26561 ts_tablet_manager.cc:1939] T 5c86a98cebc2484696b37c911b3cc050 P d7632b2891444502858ae543d072076e: Deleting consensus metadata
I20260618 22:33:36.002722 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:35.968476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43082 (local address 127.18.150.129:43405)
0618 22:33:35.968662 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:35.968668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:35.979727 (+ 11059us) server_negotiation.cc:407] Connection header received
0618 22:33:35.980604 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:35.980610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:35.980714 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:35.980793 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:35.984451 (+ 3658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.985698 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.990395 (+ 4697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:35.991038 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:35.991142 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:35.995591 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:35.995608 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:35.995629 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:35.995676 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:35.999483 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:36.000415 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:36.000424 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:36.000427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:36.000618 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:36.001023 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:36.001028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:36.001032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:36.001455 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:36.001759 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:36.002123 (+ 364us) server_negotiation.cc:299] Negotiation successful
0618 22:33:36.002355 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260618 22:33:36.003408 26145 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:36.003703 25954 catalog_manager.cc:4841] TS d7632b2891444502858ae543d072076e (127.18.150.130:37773): tablet 5c86a98cebc2484696b37c911b3cc050 (table test-table [id=a816dfc9f5994a6d92fe61bdc32496d7]) successfully deleted
I20260618 22:33:36.118705 26563 ts_tablet_manager.cc:1918] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:36.118948 26563 log.cc:1196] T 5c86a98cebc2484696b37c911b3cc050 P d3b54272febb41a292c6bf632baaedbc: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/5c86a98cebc2484696b37c911b3cc050
I20260618 22:33:36.119532 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 25951
I20260618 22:33:36.133770 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 26162
I20260618 22:33:36.134083 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.133605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43405 (local address 127.0.0.1:43084)
0618 22:33:36.133790 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:36.133893 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260618 22:33:36.134379 25956 catalog_manager.cc:4584] TS d3b54272febb41a292c6bf632baaedbc (127.18.150.129:43405): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c86a98cebc2484696b37c911b3cc050: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43405: connect: Connection refused (error 111)
I20260618 22:33:36.145968 26533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.134520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.18.150.130:37773)
0618 22:33:36.134690 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:36.134696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:36.134718 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:33:36.134839 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:36.134845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:36.134917 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:36.135001 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:36.136185 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:36.136901 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:36.138050 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:36.138440 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:36.138548 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:36.140797 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:36.140811 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:36.140826 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:36.140865 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:36.143461 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:36.144056 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:36.144063 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:36.144066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:36.144195 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:36.144574 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:36.144578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:36.144580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:36.144910 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:36.145187 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:36.145435 (+ 248us) server_negotiation.cc:299] Negotiation successful
0618 22:33:36.145630 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:33:36.146765 26292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:36.195052 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.194455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43405 (local address 127.0.0.1:43086)
0618 22:33:36.194703 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:36.194853 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260618 22:33:36.196098 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.195730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43405 (local address 127.0.0.1:43094)
0618 22:33:36.195912 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:36.195980 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260618 22:33:36.239017 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 26162
I20260618 22:33:36.252458 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 26310
I20260618 22:33:36.264531 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.264036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43405 (local address 127.0.0.1:43110)
0618 22:33:36.264251 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:36.264370 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260618 22:33:36.265532 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.265186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43405 (local address 127.0.0.1:43122)
0618 22:33:36.265335 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:36.265409 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260618 22:33:36.266753 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.253285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47008 (local address 127.18.150.131:38539)
0618 22:33:36.253474 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:36.253481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:36.253555 (+ 74us) server_negotiation.cc:407] Connection header received
0618 22:33:36.253625 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:36.253632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:36.253714 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:36.253845 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:36.255104 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:36.256266 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:36.257494 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:36.258323 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:36.258459 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:36.260806 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:36.260829 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:36.260850 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:36.260893 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:36.264066 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:36.264776 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:36.264787 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:36.264790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:36.264955 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:36.265313 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:36.265318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:36.265321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:36.265710 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:36.266118 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:36.266257 (+ 139us) server_negotiation.cc:299] Negotiation successful
0618 22:33:36.266457 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:33:36.267326 26433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:36.335317 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.334931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:32910)
0618 22:33:36.335098 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:36.335182 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260618 22:33:36.336175 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.335896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35785 (local address 127.0.0.1:32914)
0618 22:33:36.336004 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:36.336076 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:35785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260618 22:33:36.367686 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.367240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43405 (local address 127.0.0.1:43126)
0618 22:33:36.367432 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:36.367530 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260618 22:33:36.368587 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.368308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43405 (local address 127.0.0.1:43132)
0618 22:33:36.368418 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:36.368483 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260618 22:33:36.381593 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 26310
I20260618 22:33:36.398125 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 25645
I20260618 22:33:36.399158 25812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:36.511389 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 25645
I20260618 22:33:36.527667 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 25746
I20260618 22:33:36.528707 26000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:36.644902 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.644381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43405 (local address 127.0.0.1:43136)
0618 22:33:36.644588 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:36.644692 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260618 22:33:36.645306 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 25746
I20260618 22:33:36.645339 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:36.644783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:37051 (local address 127.0.0.1:53340)
0618 22:33:36.645104 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:36.645179 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:37051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"threads_started":1}
2026-06-18T22:33:36Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9302 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:33:36 dist-test-slave-h7jz krb5kdc[26573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:33:36 dist-test-slave-h7jz krb5kdc[26573](info): set up 2 sockets
Jun 18 22:33:36 dist-test-slave-h7jz krb5kdc[26573](info): commencing operation
krb5kdc: starting...
W20260618 22:33:37.650568 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.006s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:33:37 dist-test-slave-h7jz krb5kdc[26573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:33:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.190.keytab.
Entry for principal oryx/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.190.keytab.
I20260618 22:33:37.851191 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33139
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.190.keytab
--principal=oryx/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:33139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:38.160156 26586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:38.160434 26586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:38.160562 26586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:38.167681 26586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:38.167783 26586 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260618 22:33:38.167824 26586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:38.167900 26586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:38.167966 26586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:38.168048 26586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:38.178160 26586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:33139
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.190.keytab
--principal=oryx/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:38.179388 26586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:38.180706 26586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:38.191817 26592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:38.191792 26594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:38.191869 26591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:38.192411 26586 server_base.cc:1034] running on GCE node
I20260618 22:33:38.193161 26586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:38.194481 26586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:38.195698 26586 hybrid_clock.cc:648] HybridClock initialized: now 1781822018195635 us; error 27 us; skew 500 ppm
I20260618 22:33:38.196023 26586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:38 dist-test-slave-h7jz krb5kdc[26573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822018, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:38.198851 26586 init.cc:377] Logged in from keytab as oryx/127.18.150.190@KRBTEST.COM (short username oryx)
I20260618 22:33:38.200145 26586 webserver.cc:460] Webserver started at http://127.18.150.190:34399/ using document root <none> and password file <none>
I20260618 22:33:38.200764 26586 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:38.200843 26586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:38.201092 26586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:38.203856 26586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "29728dc62c584cce9d03adf34ec39f48"
format_stamp: "Formatted at 2026-06-18 22:33:38 on dist-test-slave-h7jz"
server_key: "f732d17ee29b0eae2d7b090d7a945539"
server_key_iv: "4026c797739246125c23f7027a64d3a5"
server_key_version: "encryptionkey@0"
I20260618 22:33:38.204658 26586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "29728dc62c584cce9d03adf34ec39f48"
format_stamp: "Formatted at 2026-06-18 22:33:38 on dist-test-slave-h7jz"
server_key: "f732d17ee29b0eae2d7b090d7a945539"
server_key_iv: "4026c797739246125c23f7027a64d3a5"
server_key_version: "encryptionkey@0"
I20260618 22:33:38.209322 26586 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260618 22:33:38.212507 26601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:38.213735 26586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260618 22:33:38.213897 26586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "29728dc62c584cce9d03adf34ec39f48"
format_stamp: "Formatted at 2026-06-18 22:33:38 on dist-test-slave-h7jz"
server_key: "f732d17ee29b0eae2d7b090d7a945539"
server_key_iv: "4026c797739246125c23f7027a64d3a5"
server_key_version: "encryptionkey@0"
I20260618 22:33:38.214118 26586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:38.243252 26586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:38.247081 26586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:38.247325 26586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:38.266528 26586 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:33139
I20260618 22:33:38.266546 26662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:33139 every 8 connection(s)
I20260618 22:33:38.267807 26586 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:38.269107 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 26586
I20260618 22:33:38.269305 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:38.270083 19034 external_mini_cluster.cc:1428] Setting key dd18fb54c8b124840751232750be7f13
I20260618 22:33:38.272226 26663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:38.282490 26663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48: Bootstrap starting.
Jun 18 22:33:38 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:38.285501 26663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:38.286610 26663 log.cc:824] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:38.289366 26663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48: No bootstrap required, opened a new log
I20260618 22:33:38.290745 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:38.272171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54496 (local address 127.18.150.190:33139)
0618 22:33:38.272888 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:38.272900 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:38.272938 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:33:38.274551 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:38.274572 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:38.274960 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:38.275323 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:38.276786 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:38.278165 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:38.279328 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:38.280003 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:38.280191 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:38.283861 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:38.283868 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:38.283880 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:38.283949 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:38.286969 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:38.287621 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:38.287629 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:38.287636 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:38.287796 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:38.288166 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:38.288169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:38.288171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:38.288600 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:38.288819 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:38.289266 (+ 447us) server_negotiation.cc:299] Negotiation successful
0618 22:33:38.289680 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":315,"threads_started":1}
I20260618 22:33:38.296466 26663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29728dc62c584cce9d03adf34ec39f48" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33139 } }
I20260618 22:33:38.296753 26663 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:38.296849 26663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29728dc62c584cce9d03adf34ec39f48, State: Initialized, Role: FOLLOWER
I20260618 22:33:38.297331 26663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29728dc62c584cce9d03adf34ec39f48" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33139 } }
I20260618 22:33:38.297518 26663 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:38.297628 26663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:38.297771 26663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:38.299746 26663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29728dc62c584cce9d03adf34ec39f48" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33139 } }
I20260618 22:33:38.300202 26663 leader_election.cc:302] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29728dc62c584cce9d03adf34ec39f48; no voters:
I20260618 22:33:38.300726 26663 leader_election.cc:288] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:38.300813 26668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:38.301465 26668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [term 1 LEADER]: Becoming Leader. State: Replica: 29728dc62c584cce9d03adf34ec39f48, State: Running, Role: LEADER
I20260618 22:33:38.301957 26668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29728dc62c584cce9d03adf34ec39f48" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33139 } }
I20260618 22:33:38.302579 26663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:38.304756 26669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29728dc62c584cce9d03adf34ec39f48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29728dc62c584cce9d03adf34ec39f48" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33139 } } }
I20260618 22:33:38.304698 26670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29728dc62c584cce9d03adf34ec39f48. Latest consensus state: current_term: 1 leader_uuid: "29728dc62c584cce9d03adf34ec39f48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29728dc62c584cce9d03adf34ec39f48" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33139 } } }
I20260618 22:33:38.305203 26670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:38.305203 26669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48 [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:38.306708 26675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:38.311786 26675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:38.320187 26675 catalog_manager.cc:1347] Generated new cluster ID: d20f99f3e6284ae6866e029853e597e3
I20260618 22:33:38.320304 26675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:38.332050 26675 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:38.333456 26675 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:38.343205 26675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48: Generated new TSK 0
I20260618 22:33:38.344089 26675 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab.
Entry for principal oryx/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab.
I20260618 22:33:38.404987 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:33139
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab
--principal=oryx/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:38.714584 26689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:38.714865 26689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:38.715000 26689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:38.722204 26689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:38.722324 26689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:38.722440 26689 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:33:38.722548 26689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:33:38.732259 26689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab
--principal=oryx/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.150.190:33139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:38.733640 26689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:38.735054 26689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:38.747169 26697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:38.747189 26694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:38.747215 26695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:38.747985 26689 server_base.cc:1034] running on GCE node
I20260618 22:33:38.748574 26689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:38.749485 26689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:38.750725 26689 hybrid_clock.cc:648] HybridClock initialized: now 1781822018750661 us; error 30 us; skew 500 ppm
I20260618 22:33:38.751015 26689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:38 dist-test-slave-h7jz krb5kdc[26573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822018, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:38.753892 26689 init.cc:377] Logged in from keytab as oryx/127.18.150.129@KRBTEST.COM (short username oryx)
I20260618 22:33:38.755205 26689 webserver.cc:460] Webserver started at http://127.18.150.129:35241/ using document root <none> and password file <none>
I20260618 22:33:38.755841 26689 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:38.755919 26689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:38.756168 26689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:38.758914 26689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "481d47b030104ed8a169d7c92652eb32"
format_stamp: "Formatted at 2026-06-18 22:33:38 on dist-test-slave-h7jz"
server_key: "ee66cca8c910bdcd4bcddf43fbf5fd0c"
server_key_iv: "7fa1d3f1bcaf1fac3357853375dd52bd"
server_key_version: "encryptionkey@0"
I20260618 22:33:38.759616 26689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "481d47b030104ed8a169d7c92652eb32"
format_stamp: "Formatted at 2026-06-18 22:33:38 on dist-test-slave-h7jz"
server_key: "ee66cca8c910bdcd4bcddf43fbf5fd0c"
server_key_iv: "7fa1d3f1bcaf1fac3357853375dd52bd"
server_key_version: "encryptionkey@0"
I20260618 22:33:38.764370 26689 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:33:38.767552 26704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:38.768795 26689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:33:38.768916 26689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "481d47b030104ed8a169d7c92652eb32"
format_stamp: "Formatted at 2026-06-18 22:33:38 on dist-test-slave-h7jz"
server_key: "ee66cca8c910bdcd4bcddf43fbf5fd0c"
server_key_iv: "7fa1d3f1bcaf1fac3357853375dd52bd"
server_key_version: "encryptionkey@0"
I20260618 22:33:38.769098 26689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:38.789011 26689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:38.793404 26689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:38.793639 26689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:38.796180 26689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:38.796281 26689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:38.796406 26689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:38.796469 26689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:33:38 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822018, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:38.820065 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:38.797592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33139 (local address 127.18.150.129:53403)
0618 22:33:38.798623 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:38.798662 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:38.800519 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:38.801251 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:38.801260 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:38.801798 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:38.802746 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:38.802784 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:38.804216 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:38.804222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:38.806849 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:38.806866 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:38.807910 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:38.807920 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:38.808137 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:38.808867 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:38.808899 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:38.811949 (+ 3050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:38.814966 (+ 3017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:38.814976 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:38.814998 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:38.815676 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:38.816064 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:38.816069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:38.816076 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:38.816366 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:38.816970 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:38.816978 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:38.817510 (+ 532us) client_negotiation.cc:769] Sending connection context
0618 22:33:38.817829 (+ 319us) client_negotiation.cc:240] Negotiation successful
0618 22:33:38.818475 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":451,"threads_started":1}
I20260618 22:33:38.821048 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:38.796804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:53403 (local address 127.18.150.190:33139)
0618 22:33:38.797207 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:38.797211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:38.798774 (+ 1563us) server_negotiation.cc:407] Connection header received
0618 22:33:38.800801 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:38.800807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:38.800890 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:38.800987 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:38.802936 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:38.804000 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:38.807022 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:38.807624 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:38.807750 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:38.812194 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:38.812201 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:38.812205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:38.812240 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:38.814832 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:38.815789 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:38.815795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:38.815798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:38.815925 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:38.816509 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:38.816514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:38.816517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:38.816846 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:38.816986 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:38.820608 (+ 3622us) server_negotiation.cc:299] Negotiation successful
0618 22:33:38.820791 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":213,"threads_started":1}
I20260618 22:33:38.836788 26689 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:37639
I20260618 22:33:38.836805 26824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:37639 every 8 connection(s)
I20260618 22:33:38.838017 26689 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:33:38.840013 26825 heartbeater.cc:347] Connected to a master server at 127.18.150.190:33139
I20260618 22:33:38.840255 26825 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:38.840385 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:38.827017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:45733 (local address 127.18.150.190:33139)
0618 22:33:38.827184 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:38.827190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:38.827885 (+ 695us) server_negotiation.cc:407] Connection header received
0618 22:33:38.828062 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:38.828065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:38.828124 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:38.828201 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:38.829317 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:38.830086 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:38.832028 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:38.832622 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:38.832715 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:38.835177 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:38.835184 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:38.835189 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:38.835230 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:38.837740 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:38.838425 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:38.838430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:38.838434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:38.838560 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:38.838978 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:38.838982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:38.838985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:38.839314 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:38.839437 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:38.839842 (+ 405us) server_negotiation.cc:299] Negotiation successful
0618 22:33:38.840062 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:33:38.840516 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:38.827092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33139 (local address 127.18.150.129:45733)
0618 22:33:38.827773 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:38.827796 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:38.827950 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:38.828243 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:38.828248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:38.828810 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:38.829186 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:38.829200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:38.830225 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:38.830229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:38.831834 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:38.831858 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:38.832990 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:38.832998 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:38.833182 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:38.833857 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:38.833876 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:38.835015 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:38.837860 (+ 2845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:38.837865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:38.837870 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:38.838322 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:38.838661 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:38.838666 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:38.838670 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:38.838835 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:38.839435 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:38.839439 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:38.839606 (+ 167us) client_negotiation.cc:769] Sending connection context
0618 22:33:38.839758 (+ 152us) client_negotiation.cc:240] Negotiation successful
0618 22:33:38.840060 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":353,"threads_started":1}
I20260618 22:33:38.841327 26825 heartbeater.cc:511] Master 127.18.150.190:33139 requested a full tablet report, sending...
I20260618 22:33:38.843554 26618 ts_manager.cc:194] Registered new tserver with Master: 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639)
I20260618 22:33:38.844909 26618 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:53403
I20260618 22:33:38.846154 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 26689
I20260618 22:33:38.846340 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:33:38.847141 19034 external_mini_cluster.cc:1428] Setting key c44ce682e33a97e761e7f569d1dfd726
WARNING: no policy specified for oryx/127.18.150.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab.
Entry for principal oryx/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab.
I20260618 22:33:38.890626 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:33139
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab
--principal=oryx/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:39.193759 26829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:39.194068 26829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:39.194213 26829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:39.201292 26829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:39.201393 26829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:39.201521 26829 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:33:39.201607 26829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:33:39.211762 26829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab
--principal=oryx/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.150.190:33139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:39.213011 26829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:39.214386 26829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:39.227478 26835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:39.227468 26834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:39.227500 26837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:39.228253 26829 server_base.cc:1034] running on GCE node
I20260618 22:33:39.228858 26829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:39.229781 26829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:39.230997 26829 hybrid_clock.cc:648] HybridClock initialized: now 1781822019230918 us; error 54 us; skew 500 ppm
I20260618 22:33:39.231285 26829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:39 dist-test-slave-h7jz krb5kdc[26573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822019, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:39.234305 26829 init.cc:377] Logged in from keytab as oryx/127.18.150.130@KRBTEST.COM (short username oryx)
I20260618 22:33:39.235637 26829 webserver.cc:460] Webserver started at http://127.18.150.130:36797/ using document root <none> and password file <none>
I20260618 22:33:39.236351 26829 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:39.236449 26829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:39.236750 26829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:39.239614 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "62859fc204084afcb527069e201ff03e"
format_stamp: "Formatted at 2026-06-18 22:33:39 on dist-test-slave-h7jz"
server_key: "fb24536912d30405b2716dc6acd64d51"
server_key_iv: "4b47c200277f7e31ca1f438ab9266a7b"
server_key_version: "encryptionkey@0"
I20260618 22:33:39.240392 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "62859fc204084afcb527069e201ff03e"
format_stamp: "Formatted at 2026-06-18 22:33:39 on dist-test-slave-h7jz"
server_key: "fb24536912d30405b2716dc6acd64d51"
server_key_iv: "4b47c200277f7e31ca1f438ab9266a7b"
server_key_version: "encryptionkey@0"
I20260618 22:33:39.245512 26829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260618 22:33:39.249038 26844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:39.250459 26829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:39.250612 26829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "62859fc204084afcb527069e201ff03e"
format_stamp: "Formatted at 2026-06-18 22:33:39 on dist-test-slave-h7jz"
server_key: "fb24536912d30405b2716dc6acd64d51"
server_key_iv: "4b47c200277f7e31ca1f438ab9266a7b"
server_key_version: "encryptionkey@0"
I20260618 22:33:39.250835 26829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:39.268371 26829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:39.272786 26829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:39.273059 26829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:39.275823 26829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:39.275910 26829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:39.276033 26829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:39.276088 26829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:33:39 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822019, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:39.301369 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.277839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:57749 (local address 127.18.150.190:33139)
0618 22:33:39.278129 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:39.278135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:39.278418 (+ 283us) server_negotiation.cc:407] Connection header received
0618 22:33:39.281546 (+ 3128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:39.281554 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:39.281683 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:39.281761 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:39.284731 (+ 2970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.285468 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.288298 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.288841 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.288981 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.294058 (+ 5077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:39.294064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:39.294067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:39.294106 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:39.297668 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:39.298801 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:39.298808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:39.298812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:39.298993 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:39.299452 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:39.299458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:39.299462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:39.299845 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:39.299997 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:39.300718 (+ 721us) server_negotiation.cc:299] Negotiation successful
0618 22:33:39.300965 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260618 22:33:39.302366 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.277252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33139 (local address 127.18.150.130:57749)
0618 22:33:39.278217 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:39.278288 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:39.281130 (+ 2842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:39.282402 (+ 1272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:39.282419 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:39.283418 (+ 999us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:39.284498 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.284534 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.285655 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.285661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.288116 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.288134 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.289145 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.289154 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.289389 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.290395 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:39.290435 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:39.293680 (+ 3245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:39.297827 (+ 4147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:39.297838 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:39.297859 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:39.298628 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:39.299124 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:39.299134 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:39.299136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:39.299348 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:39.299977 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:39.299984 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:39.300375 (+ 391us) client_negotiation.cc:769] Sending connection context
0618 22:33:39.300602 (+ 227us) client_negotiation.cc:240] Negotiation successful
0618 22:33:39.301007 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":788,"threads_started":1}
I20260618 22:33:39.322620 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.308381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:39399 (local address 127.18.150.190:33139)
0618 22:33:39.308555 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:39.308560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:39.309085 (+ 525us) server_negotiation.cc:407] Connection header received
0618 22:33:39.309245 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:39.309251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:39.309319 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:39.309397 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:39.310482 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.311595 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.313357 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.313905 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.314061 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.316004 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:39.316010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:39.316013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:39.316048 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:39.319160 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:39.320137 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:39.320144 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:39.320148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:39.320331 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:39.320779 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:39.320784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:39.320789 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:39.321267 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:39.321423 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:39.321936 (+ 513us) server_negotiation.cc:299] Negotiation successful
0618 22:33:39.322197 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:33:39.322703 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.308437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33139 (local address 127.18.150.130:39399)
0618 22:33:39.308987 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:39.309002 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:39.309133 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:39.309432 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:39.309437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:39.309886 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:39.310261 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.310272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.311741 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.311746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.313157 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.313176 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.314115 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.314121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.314264 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.314929 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:39.314948 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:39.315788 (+ 840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:39.319308 (+ 3520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:39.319315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:39.319320 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:39.319984 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:39.320465 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:39.320471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:39.320475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:39.320661 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:39.321390 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:39.321396 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:39.321642 (+ 246us) client_negotiation.cc:769] Sending connection context
0618 22:33:39.321822 (+ 180us) client_negotiation.cc:240] Negotiation successful
0618 22:33:39.322205 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":304,"threads_started":1}
I20260618 22:33:39.329288 26829 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:41033
I20260618 22:33:39.329332 26963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:41033 every 8 connection(s)
I20260618 22:33:39.330504 26829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:33:39.332580 26964 heartbeater.cc:347] Connected to a master server at 127.18.150.190:33139
I20260618 22:33:39.332803 26964 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:39.333432 26964 heartbeater.cc:511] Master 127.18.150.190:33139 requested a full tablet report, sending...
I20260618 22:33:39.334784 26618 ts_manager.cc:194] Registered new tserver with Master: 62859fc204084afcb527069e201ff03e (127.18.150.130:41033)
I20260618 22:33:39.335587 26618 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:57749
I20260618 22:33:39.339231 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 26829
I20260618 22:33:39.339371 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:33:39.339773 19034 external_mini_cluster.cc:1428] Setting key d10e794338f92e2f985b47ec86fc677b
WARNING: no policy specified for oryx/127.18.150.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab.
Entry for principal oryx/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab.
I20260618 22:33:39.383025 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:33139
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab
--principal=oryx/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:39.691720 26968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:39.692001 26968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:39.692093 26968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:39.699167 26968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:39.699280 26968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:39.699360 26968 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:33:39.699437 26968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:33:39.709374 26968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab
--principal=oryx/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.150.190:33139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:39.710613 26968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:39.711905 26968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:39.724627 26976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:39.724722 26973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:39.724685 26974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:39.726084 26968 server_base.cc:1034] running on GCE node
I20260618 22:33:39.727176 26968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:39.728320 26968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:39.729552 26968 hybrid_clock.cc:648] HybridClock initialized: now 1781822019729464 us; error 59 us; skew 500 ppm
I20260618 22:33:39.729918 26968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:39 dist-test-slave-h7jz krb5kdc[26573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822019, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:39.733076 26968 init.cc:377] Logged in from keytab as oryx/127.18.150.131@KRBTEST.COM (short username oryx)
I20260618 22:33:39.734524 26968 webserver.cc:460] Webserver started at http://127.18.150.131:41187/ using document root <none> and password file <none>
I20260618 22:33:39.735214 26968 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:39.735314 26968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:39.735618 26968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:39.738524 26968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240"
format_stamp: "Formatted at 2026-06-18 22:33:39 on dist-test-slave-h7jz"
server_key: "96d38639fd2cebddc5db0f972a0f319b"
server_key_iv: "831d0d883eee55cc929926cfcd72c658"
server_key_version: "encryptionkey@0"
I20260618 22:33:39.739274 26968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240"
format_stamp: "Formatted at 2026-06-18 22:33:39 on dist-test-slave-h7jz"
server_key: "96d38639fd2cebddc5db0f972a0f319b"
server_key_iv: "831d0d883eee55cc929926cfcd72c658"
server_key_version: "encryptionkey@0"
I20260618 22:33:39.744315 26968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:33:39.747789 26983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:39.749126 26968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:33:39.749277 26968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240"
format_stamp: "Formatted at 2026-06-18 22:33:39 on dist-test-slave-h7jz"
server_key: "96d38639fd2cebddc5db0f972a0f319b"
server_key_iv: "831d0d883eee55cc929926cfcd72c658"
server_key_version: "encryptionkey@0"
I20260618 22:33:39.749481 26968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:39.770949 26968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:39.775041 26968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:39.775292 26968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:39.777799 26968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:39.777882 26968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:39.778036 26968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:39.778092 26968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:33:39 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822019, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:39.802789 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.778489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:37361 (local address 127.18.150.190:33139)
0618 22:33:39.778669 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:39.778675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:39.780150 (+ 1475us) server_negotiation.cc:407] Connection header received
0618 22:33:39.782591 (+ 2441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:39.782598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:39.782694 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:39.782810 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:39.785139 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.786316 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.788645 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.789260 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.789378 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.794299 (+ 4921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:39.794305 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:39.794308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:39.794343 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:39.797865 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:39.798834 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:39.798840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:39.798844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:39.799020 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:39.799628 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:39.799634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:39.799638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:39.799966 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:39.800117 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:39.802181 (+ 2064us) server_negotiation.cc:299] Negotiation successful
0618 22:33:39.802444 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:33:39.803261 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.779111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33139 (local address 127.18.150.131:37361)
0618 22:33:39.779946 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:39.780001 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:39.782235 (+ 2234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:39.783123 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:39.783139 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:39.783945 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:39.784857 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.784878 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.786514 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.786521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.788440 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.788452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.789660 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.789665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.789987 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.790811 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:39.790858 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:39.794043 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:39.798063 (+ 4020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:39.798072 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:39.798087 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:39.798695 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:39.799159 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:39.799170 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:39.799174 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:39.799493 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:39.800205 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:39.800215 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:39.800713 (+ 498us) client_negotiation.cc:769] Sending connection context
0618 22:33:39.801067 (+ 354us) client_negotiation.cc:240] Negotiation successful
0618 22:33:39.801712 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":328,"threads_started":1}
I20260618 22:33:39.820461 26968 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:37527
I20260618 22:33:39.820478 27102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:37527 every 8 connection(s)
I20260618 22:33:39.822146 26968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:33:39.822248 27076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.808771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33139 (local address 127.18.150.131:55577)
0618 22:33:39.809406 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:39.809431 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:39.809581 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:39.809897 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:39.809902 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:39.810608 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:39.811005 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.811018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.812133 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.812140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.813749 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.813763 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.814818 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.814824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.814939 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.815568 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:39.815584 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:39.816251 (+ 667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:39.819736 (+ 3485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:39.819742 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:39.819749 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:39.820175 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:39.820590 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:39.820595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:39.820598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:39.820750 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:39.821302 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:39.821305 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:39.821457 (+ 152us) client_negotiation.cc:769] Sending connection context
0618 22:33:39.821563 (+ 106us) client_negotiation.cc:240] Negotiation successful
0618 22:33:39.821844 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":314,"threads_started":1}
I20260618 22:33:39.822324 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.808683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:55577 (local address 127.18.150.190:33139)
0618 22:33:39.808848 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:39.808854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:39.809520 (+ 666us) server_negotiation.cc:407] Connection header received
0618 22:33:39.809699 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:39.809703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:39.809773 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:39.809852 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:39.811159 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.811989 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.813966 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.814622 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.814769 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.816460 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:39.816467 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:39.816473 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:39.816515 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:39.819573 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:39.820306 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:39.820312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:39.820315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:39.820469 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:39.820848 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:39.820853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:39.820856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:39.821198 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:39.821341 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:39.821731 (+ 390us) server_negotiation.cc:299] Negotiation successful
0618 22:33:39.821980 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:33:39.824159 27103 heartbeater.cc:347] Connected to a master server at 127.18.150.190:33139
I20260618 22:33:39.824769 27103 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:39.825479 27103 heartbeater.cc:511] Master 127.18.150.190:33139 requested a full tablet report, sending...
I20260618 22:33:39.826892 26618 ts_manager.cc:194] Registered new tserver with Master: d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131:37527)
I20260618 22:33:39.827637 26618 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:37361
I20260618 22:33:39.832186 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 26968
I20260618 22:33:39.832317 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:33:39.832680 19034 external_mini_cluster.cc:1428] Setting key bcf9ac13d706c1f7eff125bd00251bb1
I20260618 22:33:39.835353 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:33:39.848002 26825 heartbeater.cc:503] Master 127.18.150.190:33139 was elected leader, sending a full tablet report...
Jun 18 22:33:39 dist-test-slave-h7jz krb5kdc[26573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822019, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:33:39 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:39 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:33:39.881664 26713 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:39.871965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40776 (local address 127.18.150.190:33139)
0618 22:33:39.872138 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:39.872142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:39.872443 (+ 301us) server_negotiation.cc:407] Connection header received
0618 22:33:39.872662 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:39.872667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:39.872735 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:39.872831 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:39.874157 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.874908 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.876086 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.876524 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.876623 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.881353 (+ 4730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40776: BlockingRecv error: recv got EOF from 127.0.0.1:40776 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 18 22:33:39 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822019, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:39.896867 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.884879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40792 (local address 127.18.150.190:33139)
0618 22:33:39.885033 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:39.885037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:39.885328 (+ 291us) server_negotiation.cc:407] Connection header received
0618 22:33:39.885473 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:39.885477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:39.885555 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:39.885640 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:39.886740 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.887405 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.888496 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.888995 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.889089 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.892430 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:39.892435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:39.892438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:39.892471 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:39.894828 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:39.895436 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:39.895440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:39.895442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:39.895559 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:39.895844 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:39.895848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:39.895850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:39.896107 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:39.896309 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:39.896460 (+ 151us) server_negotiation.cc:299] Negotiation successful
0618 22:33:39.896622 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:33:39.900455 26618 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:33:39.907379 26618 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:33:39.940882 27122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.918876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57678 (local address 127.18.150.129:37639)
0618 22:33:39.933093 (+ 14217us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:39.933101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:39.933128 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:33:39.933517 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:39.933524 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:39.933769 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:39.933988 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:39.934579 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.935585 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.938285 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.939694 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.939855 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.940019 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:39.940415 (+ 396us) server_negotiation.cc:299] Negotiation successful
0618 22:33:39.940585 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14057,"thread_start_us":14311,"threads_started":1}
I20260618 22:33:39.941068 27124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.918460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:37639 (local address 127.0.0.1:57678)
0618 22:33:39.931094 (+ 12634us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:39.931144 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:39.931361 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:39.933988 (+ 2627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:39.933993 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:39.934018 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:39.934429 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.934437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.935759 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.935764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.938132 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.938151 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.939906 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.939912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.940106 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.940227 (+ 121us) client_negotiation.cc:769] Sending connection context
0618 22:33:39.940344 (+ 117us) client_negotiation.cc:240] Negotiation successful
0618 22:33:39.940590 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12368,"mutex_wait_us":68,"thread_start_us":13770,"threads_started":1}
I20260618 22:33:39.941450 27123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.919032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:37527 (local address 127.0.0.1:42052)
0618 22:33:39.932787 (+ 13755us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:39.932803 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:39.932906 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:39.933488 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:39.933496 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:39.933526 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:39.933986 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.934004 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.936062 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.936066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.938322 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.938331 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.939988 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.939993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.940152 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.940222 (+ 70us) client_negotiation.cc:769] Sending connection context
0618 22:33:39.940344 (+ 122us) client_negotiation.cc:240] Negotiation successful
0618 22:33:39.940590 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13639,"mutex_wait_us":143,"thread_start_us":11636,"threads_started":1}
I20260618 22:33:39.941605 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.919585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42052 (local address 127.18.150.131:37527)
0618 22:33:39.932078 (+ 12493us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:39.932087 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:39.933011 (+ 924us) server_negotiation.cc:407] Connection header received
0618 22:33:39.933099 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:39.933105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:39.933296 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:39.933474 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:39.934235 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.935926 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.938449 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.939516 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.939687 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.939862 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:39.941149 (+ 1287us) server_negotiation.cc:299] Negotiation successful
0618 22:33:39.941318 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12325,"thread_start_us":12152,"threads_started":1}
I20260618 22:33:39.941519 27121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.918987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:41033 (local address 127.0.0.1:44714)
0618 22:33:39.932526 (+ 13539us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:39.932549 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:39.932695 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:39.933707 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:39.933712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:39.933746 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:39.934234 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.934245 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.936414 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.936419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.938163 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.938175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.940746 (+ 2571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.940752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.940925 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.940959 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:33:39.941038 (+ 79us) client_negotiation.cc:240] Negotiation successful
0618 22:33:39.941161 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13382,"mutex_wait_us":21,"thread_start_us":13532,"threads_started":1}
I20260618 22:33:39.942080 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.919412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.18.150.130:41033)
0618 22:33:39.932352 (+ 12940us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:39.932360 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:39.932849 (+ 489us) server_negotiation.cc:407] Connection header received
0618 22:33:39.932959 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:39.932968 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:39.933263 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:39.933474 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:39.934405 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.935887 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.938342 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.940294 (+ 1952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.940445 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.940598 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:39.941645 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0618 22:33:39.941815 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12783,"thread_start_us":12685,"threads_started":1}
I20260618 22:33:39.945092 26755 tablet_service.cc:1467] Processing CreateTablet for tablet e9fab5e8737e47dba2621cfecd93a511 (DEFAULT_TABLE table=test-table [id=b103e9f8ebea4e3d95da6c7ada1f8628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:39.945425 26894 tablet_service.cc:1467] Processing CreateTablet for tablet e9fab5e8737e47dba2621cfecd93a511 (DEFAULT_TABLE table=test-table [id=b103e9f8ebea4e3d95da6c7ada1f8628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:39.946631 26755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9fab5e8737e47dba2621cfecd93a511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:39.946673 26894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9fab5e8737e47dba2621cfecd93a511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:39.950623 27033 tablet_service.cc:1467] Processing CreateTablet for tablet e9fab5e8737e47dba2621cfecd93a511 (DEFAULT_TABLE table=test-table [id=b103e9f8ebea4e3d95da6c7ada1f8628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:39.951712 27033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9fab5e8737e47dba2621cfecd93a511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:39.960616 27127 tablet_bootstrap.cc:492] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32: Bootstrap starting.
I20260618 22:33:39.961724 27128 tablet_bootstrap.cc:492] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e: Bootstrap starting.
I20260618 22:33:39.966131 27127 tablet_bootstrap.cc:654] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:39.966993 27129 tablet_bootstrap.cc:492] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240: Bootstrap starting.
I20260618 22:33:39.967451 27127 log.cc:824] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:39.969156 27128 tablet_bootstrap.cc:654] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:39.970001 27127 tablet_bootstrap.cc:492] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32: No bootstrap required, opened a new log
I20260618 22:33:39.970239 27127 ts_tablet_manager.cc:1397] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260618 22:33:39.970688 27129 tablet_bootstrap.cc:654] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:39.970671 27128 log.cc:824] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:39.971956 27129 log.cc:824] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:39.974709 27129 tablet_bootstrap.cc:492] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240: No bootstrap required, opened a new log
I20260618 22:33:39.974944 27129 ts_tablet_manager.cc:1397] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260618 22:33:39.975061 27128 tablet_bootstrap.cc:492] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e: No bootstrap required, opened a new log
I20260618 22:33:39.975320 27128 ts_tablet_manager.cc:1397] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.000s
I20260618 22:33:39.980204 27127 raft_consensus.cc:348] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:39.980646 27127 raft_consensus.cc:374] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:39.980739 27127 raft_consensus.cc:729] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 481d47b030104ed8a169d7c92652eb32, State: Initialized, Role: FOLLOWER
I20260618 22:33:39.981807 27127 consensus_queue.cc:260] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:39.981765 27129 raft_consensus.cc:348] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:39.982162 27129 raft_consensus.cc:374] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:39.982249 27129 raft_consensus.cc:729] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f1d9712ada46ab9a8d4c5ef8c88240, State: Initialized, Role: FOLLOWER
I20260618 22:33:39.982843 27129 consensus_queue.cc:260] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:39.983732 27127 ts_tablet_manager.cc:1428] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260618 22:33:39.984766 27103 heartbeater.cc:503] Master 127.18.150.190:33139 was elected leader, sending a full tablet report...
I20260618 22:33:39.984998 27128 raft_consensus.cc:348] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:39.985381 27128 raft_consensus.cc:374] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:39.985430 27129 ts_tablet_manager.cc:1428] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
I20260618 22:33:39.985466 27128 raft_consensus.cc:729] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62859fc204084afcb527069e201ff03e, State: Initialized, Role: FOLLOWER
I20260618 22:33:39.986017 27128 consensus_queue.cc:260] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:39.986984 26964 heartbeater.cc:503] Master 127.18.150.190:33139 was elected leader, sending a full tablet report...
I20260618 22:33:39.987612 27128 ts_tablet_manager.cc:1428] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260618 22:33:39.990306 27135 raft_consensus.cc:482] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:33:39.990499 27135 raft_consensus.cc:504] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:39.991931 27135 leader_election.cc:288] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639), d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131:37527)
I20260618 22:33:39.999333 27122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.992293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:41421 (local address 127.18.150.129:37639)
0618 22:33:39.992523 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:39.992528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:39.993220 (+ 692us) server_negotiation.cc:407] Connection header received
0618 22:33:39.993365 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:39.993370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:39.993432 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:39.993510 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:39.994327 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.995008 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.997630 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.998458 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.998574 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.998626 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:39.998978 (+ 352us) server_negotiation.cc:299] Negotiation successful
0618 22:33:39.999095 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260618 22:33:39.999459 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.992313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:37639 (local address 127.18.150.130:41421)
0618 22:33:39.993152 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:39.993165 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:39.993262 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:39.993522 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:39.993526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:39.993558 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:39.994138 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.994155 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.995156 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.995162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.997490 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.997502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.998650 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.998656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.998830 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.998866 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:33:39.998964 (+ 98us) client_negotiation.cc:240] Negotiation successful
0618 22:33:39.999102 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":113,"spinlock_wait_cycles":26240,"thread_start_us":585,"threads_started":1}
I20260618 22:33:39.999852 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.992337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:47305 (local address 127.18.150.131:37527)
0618 22:33:39.992491 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:39.992497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:39.993123 (+ 626us) server_negotiation.cc:407] Connection header received
0618 22:33:39.993317 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:39.993322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:39.993393 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:39.993487 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:39.994320 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.995238 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.998039 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.998953 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.999089 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.999152 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:39.999526 (+ 374us) server_negotiation.cc:299] Negotiation successful
0618 22:33:39.999631 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:33:40.000478 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:39.992280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:37527 (local address 127.18.150.130:47305)
0618 22:33:39.993014 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:39.993037 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:39.993173 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:39.993511 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:39.993515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:39.993556 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:39.994166 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.994175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.995354 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.995359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.997488 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:39.997502 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:39.999151 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:39.999157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:39.999335 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:39.999365 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:33:40.000005 (+ 640us) client_negotiation.cc:240] Negotiation successful
0618 22:33:40.000134 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":96,"spinlock_wait_cycles":9472,"thread_start_us":332,"threads_started":1}
I20260618 22:33:40.000425 26775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fab5e8737e47dba2621cfecd93a511" candidate_uuid: "62859fc204084afcb527069e201ff03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481d47b030104ed8a169d7c92652eb32" is_pre_election: true
I20260618 22:33:40.000969 26775 raft_consensus.cc:2454] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62859fc204084afcb527069e201ff03e in term 0.
I20260618 22:33:40.001274 27053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fab5e8737e47dba2621cfecd93a511" candidate_uuid: "62859fc204084afcb527069e201ff03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" is_pre_election: true
I20260618 22:33:40.001590 26846 leader_election.cc:302] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 481d47b030104ed8a169d7c92652eb32, 62859fc204084afcb527069e201ff03e; no voters:
I20260618 22:33:40.001794 27053 raft_consensus.cc:2454] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62859fc204084afcb527069e201ff03e in term 0.
I20260618 22:33:40.002087 27135 raft_consensus.cc:2781] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:33:40.002239 27135 raft_consensus.cc:482] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:33:40.002362 27135 raft_consensus.cc:3037] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:40.004626 27135 raft_consensus.cc:504] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.005198 27135 leader_election.cc:288] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [CANDIDATE]: Term 1 election: Requested vote from peers 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639), d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131:37527)
I20260618 22:33:40.005578 27053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fab5e8737e47dba2621cfecd93a511" candidate_uuid: "62859fc204084afcb527069e201ff03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240"
I20260618 22:33:40.005595 26775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fab5e8737e47dba2621cfecd93a511" candidate_uuid: "62859fc204084afcb527069e201ff03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481d47b030104ed8a169d7c92652eb32"
I20260618 22:33:40.005740 27053 raft_consensus.cc:3037] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:40.005769 26775 raft_consensus.cc:3037] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:40.007961 27053 raft_consensus.cc:2454] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62859fc204084afcb527069e201ff03e in term 1.
I20260618 22:33:40.008082 26775 raft_consensus.cc:2454] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62859fc204084afcb527069e201ff03e in term 1.
I20260618 22:33:40.008308 26847 leader_election.cc:302] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62859fc204084afcb527069e201ff03e, d2f1d9712ada46ab9a8d4c5ef8c88240; no voters:
I20260618 22:33:40.008622 27135 raft_consensus.cc:2781] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:40.009025 27135 raft_consensus.cc:686] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 1 LEADER]: Becoming Leader. State: Replica: 62859fc204084afcb527069e201ff03e, State: Running, Role: LEADER
I20260618 22:33:40.009536 27135 consensus_queue.cc:237] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.013674 26616 catalog_manager.cc:5495] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62859fc204084afcb527069e201ff03e (127.18.150.130). New cstate: current_term: 1 leader_uuid: "62859fc204084afcb527069e201ff03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } health_report { overall_health: UNKNOWN } } }
I20260618 22:33:40.041381 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.036718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44718 (local address 127.18.150.130:41033)
0618 22:33:40.036942 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:40.036947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:40.036964 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:33:40.037091 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:40.037094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:40.037170 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:40.037294 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:33:40.037569 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.038349 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.039541 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.039933 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.040015 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.040342 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:33:40.040876 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:33:40.040983 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:40.041055 (+ 72us) server_negotiation.cc:299] Negotiation successful
0618 22:33:40.041118 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260618 22:33:40.049042 27053 raft_consensus.cc:1261] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Refusing update from remote peer 62859fc204084afcb527069e201ff03e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:40.049052 26775 raft_consensus.cc:1261] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Refusing update from remote peer 62859fc204084afcb527069e201ff03e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:40.049989 27138 consensus_queue.cc:1035] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:40.050304 27135 consensus_queue.cc:1035] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:40.063452 27141 mvcc.cc:205] Tried to move back new op lower bound from 7298342994112274432 to 7298342993963278336. Current Snapshot: MvccSnapshot[applied={T|T < 7298342994112274432}]
I20260618 22:33:40.080634 26616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:40.080935 26616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:40.084667 26616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48: Sending DeleteTablet for 3 replicas of tablet e9fab5e8737e47dba2621cfecd93a511
I20260618 22:33:40.085711 26755 tablet_service.cc:1514] Processing DeleteTablet for tablet e9fab5e8737e47dba2621cfecd93a511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:40 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:57678
I20260618 22:33:40.086066 27033 tablet_service.cc:1514] Processing DeleteTablet for tablet e9fab5e8737e47dba2621cfecd93a511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:40 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:42052
I20260618 22:33:40.086067 26894 tablet_service.cc:1514] Processing DeleteTablet for tablet e9fab5e8737e47dba2621cfecd93a511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:40 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:44714
I20260618 22:33:40.089426 26616 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:33:40.092156 26616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:33:40.096827 27152 tablet_replica.cc:330] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32: stopping tablet replica
I20260618 22:33:40.097291 27152 raft_consensus.cc:2229] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:40.097785 27152 raft_consensus.cc:2258] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:40.098546 27154 tablet_replica.cc:330] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e: stopping tablet replica
I20260618 22:33:40.098989 27154 raft_consensus.cc:2229] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:33:40.099704 27154 raft_consensus.cc:2258] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:40.100776 27153 tablet_replica.cc:330] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240: stopping tablet replica
I20260618 22:33:40.101179 27153 raft_consensus.cc:2229] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:40.101620 27153 raft_consensus.cc:2258] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:40.103147 26755 tablet_service.cc:1467] Processing CreateTablet for tablet 828e4658a5564f2da9f603a9a66a105c (DEFAULT_TABLE table=test-table [id=b0a326b31ab6409e9021910facf99239]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:40.103156 26894 tablet_service.cc:1467] Processing CreateTablet for tablet 828e4658a5564f2da9f603a9a66a105c (DEFAULT_TABLE table=test-table [id=b0a326b31ab6409e9021910facf99239]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:40.103621 26755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828e4658a5564f2da9f603a9a66a105c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:40.103619 26894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828e4658a5564f2da9f603a9a66a105c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:40.104624 27154 ts_tablet_manager.cc:1905] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:40.104373 27152 ts_tablet_manager.cc:1905] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:40.110337 27128 tablet_bootstrap.cc:492] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e: Bootstrap starting.
I20260618 22:33:40.110935 27033 tablet_service.cc:1467] Processing CreateTablet for tablet 828e4658a5564f2da9f603a9a66a105c (DEFAULT_TABLE table=test-table [id=b0a326b31ab6409e9021910facf99239]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:40.111393 27033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828e4658a5564f2da9f603a9a66a105c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:40.112010 27128 tablet_bootstrap.cc:654] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:40.114001 27127 tablet_bootstrap.cc:492] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32: Bootstrap starting.
I20260618 22:33:40.115628 27127 tablet_bootstrap.cc:654] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:40.116755 27129 tablet_bootstrap.cc:492] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240: Bootstrap starting.
I20260618 22:33:40.117991 27129 tablet_bootstrap.cc:654] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:40.120016 27129 tablet_bootstrap.cc:492] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240: No bootstrap required, opened a new log
I20260618 22:33:40.120142 27129 ts_tablet_manager.cc:1397] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260618 22:33:40.120728 27129 raft_consensus.cc:348] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.120894 27129 raft_consensus.cc:374] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:40.120983 27129 raft_consensus.cc:729] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f1d9712ada46ab9a8d4c5ef8c88240, State: Initialized, Role: FOLLOWER
I20260618 22:33:40.121219 27129 consensus_queue.cc:260] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.122901 27129 ts_tablet_manager.cc:1428] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260618 22:33:40.126199 27153 ts_tablet_manager.cc:1905] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:40.128531 27152 ts_tablet_manager.cc:1918] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:40.128741 27152 log.cc:1196] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/e9fab5e8737e47dba2621cfecd93a511
I20260618 22:33:40.129325 27152 ts_tablet_manager.cc:1939] T e9fab5e8737e47dba2621cfecd93a511 P 481d47b030104ed8a169d7c92652eb32: Deleting consensus metadata
I20260618 22:33:40.131475 27128 tablet_bootstrap.cc:492] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e: No bootstrap required, opened a new log
I20260618 22:33:40.131652 27128 ts_tablet_manager.cc:1397] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.005s
I20260618 22:33:40.131702 27153 ts_tablet_manager.cc:1918] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:40.131858 27153 log.cc:1196] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/e9fab5e8737e47dba2621cfecd93a511
I20260618 22:33:40.132328 27153 ts_tablet_manager.cc:1939] T e9fab5e8737e47dba2621cfecd93a511 P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting consensus metadata
I20260618 22:33:40.132557 27154 ts_tablet_manager.cc:1918] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:40.132507 27128 raft_consensus.cc:348] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.132771 27128 raft_consensus.cc:374] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:40.132877 27128 raft_consensus.cc:729] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62859fc204084afcb527069e201ff03e, State: Initialized, Role: FOLLOWER
I20260618 22:33:40.132879 27154 log.cc:1196] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/e9fab5e8737e47dba2621cfecd93a511
I20260618 22:33:40.133211 27128 consensus_queue.cc:260] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.133602 27154 ts_tablet_manager.cc:1939] T e9fab5e8737e47dba2621cfecd93a511 P 62859fc204084afcb527069e201ff03e: Deleting consensus metadata
I20260618 22:33:40.133898 27128 ts_tablet_manager.cc:1428] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:33:40.134153 26604 catalog_manager.cc:4841] TS d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131:37527): tablet e9fab5e8737e47dba2621cfecd93a511 (table test-table [id=b103e9f8ebea4e3d95da6c7ada1f8628]) successfully deleted
I20260618 22:33:40.135062 26603 catalog_manager.cc:4841] TS 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639): tablet e9fab5e8737e47dba2621cfecd93a511 (table test-table [id=b103e9f8ebea4e3d95da6c7ada1f8628]) successfully deleted
I20260618 22:33:40.135596 26605 catalog_manager.cc:4841] TS 62859fc204084afcb527069e201ff03e (127.18.150.130:41033): tablet e9fab5e8737e47dba2621cfecd93a511 (table test-table [id=b103e9f8ebea4e3d95da6c7ada1f8628]) successfully deleted
I20260618 22:33:40.136376 27127 tablet_bootstrap.cc:492] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32: No bootstrap required, opened a new log
I20260618 22:33:40.136528 27127 ts_tablet_manager.cc:1397] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.007s
I20260618 22:33:40.137311 27127 raft_consensus.cc:348] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.137533 27127 raft_consensus.cc:374] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:40.137614 27127 raft_consensus.cc:729] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 481d47b030104ed8a169d7c92652eb32, State: Initialized, Role: FOLLOWER
I20260618 22:33:40.137900 27127 consensus_queue.cc:260] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.138520 27127 ts_tablet_manager.cc:1428] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260618 22:33:40.325691 27104 tablet.cc:2369] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:33:40.333261 26965 tablet.cc:2369] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:33:40.334774 27133 raft_consensus.cc:482] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:33:40.334970 27133 raft_consensus.cc:504] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.336594 27133 leader_election.cc:288] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62859fc204084afcb527069e201ff03e (127.18.150.130:41033), d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131:37527)
W20260618 22:33:40.341439 26826 tablet.cc:2369] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:33:40.350265 27138 raft_consensus.cc:482] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:33:40.350492 27138 raft_consensus.cc:504] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.351243 27138 leader_election.cc:288] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639), d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131:37527)
I20260618 22:33:40.352371 26775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828e4658a5564f2da9f603a9a66a105c" candidate_uuid: "62859fc204084afcb527069e201ff03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481d47b030104ed8a169d7c92652eb32" is_pre_election: true
I20260618 22:33:40.352608 26775 raft_consensus.cc:2454] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62859fc204084afcb527069e201ff03e in term 0.
I20260618 22:33:40.352963 27053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828e4658a5564f2da9f603a9a66a105c" candidate_uuid: "62859fc204084afcb527069e201ff03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" is_pre_election: true
I20260618 22:33:40.353193 27053 raft_consensus.cc:2454] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62859fc204084afcb527069e201ff03e in term 0.
I20260618 22:33:40.353788 26846 leader_election.cc:302] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 481d47b030104ed8a169d7c92652eb32, 62859fc204084afcb527069e201ff03e; no voters:
I20260618 22:33:40.354178 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.337395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:52591 (local address 127.18.150.130:41033)
0618 22:33:40.337588 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:40.337595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:40.347126 (+ 9531us) server_negotiation.cc:407] Connection header received
0618 22:33:40.347231 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:40.347237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:40.347338 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:40.347440 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:40.348258 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.349407 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.351747 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.352946 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.353085 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.353163 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:40.353726 (+ 563us) server_negotiation.cc:299] Negotiation successful
0618 22:33:40.353873 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:33:40.354837 27138 raft_consensus.cc:2781] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:33:40.354878 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.336836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:41033 (local address 127.18.150.129:52591)
0618 22:33:40.346761 (+ 9925us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:40.346788 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:40.346981 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:40.347654 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:40.347659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:40.347702 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:40.348068 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.348080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.349582 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.349587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.351569 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.351583 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.353398 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.353404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.353566 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.353597 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:33:40.354382 (+ 785us) client_negotiation.cc:240] Negotiation successful
0618 22:33:40.354523 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9746,"thread_start_us":10237,"threads_started":1}
I20260618 22:33:40.354943 27138 raft_consensus.cc:482] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:33:40.355057 27138 raft_consensus.cc:3037] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:40.355908 26918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828e4658a5564f2da9f603a9a66a105c" candidate_uuid: "481d47b030104ed8a169d7c92652eb32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62859fc204084afcb527069e201ff03e" is_pre_election: true
I20260618 22:33:40.358343 27138 raft_consensus.cc:504] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.358811 26918 raft_consensus.cc:2379] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 481d47b030104ed8a169d7c92652eb32 in current term 1: Already voted for candidate 62859fc204084afcb527069e201ff03e in this term.
I20260618 22:33:40.360047 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.337589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:59871 (local address 127.18.150.131:37527)
0618 22:33:40.337768 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:40.337774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:40.347822 (+ 10048us) server_negotiation.cc:407] Connection header received
0618 22:33:40.348120 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:40.348126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:40.348203 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:40.348261 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:40.348957 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.350000 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.357727 (+ 7727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.358861 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.358985 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.359060 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:40.359668 (+ 608us) server_negotiation.cc:299] Negotiation successful
0618 22:33:40.359803 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:33:40.360760 27159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.336986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:37527 (local address 127.18.150.129:59871)
0618 22:33:40.347712 (+ 10726us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:40.347728 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:40.347925 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:40.348443 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:40.348447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:40.348472 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:40.348794 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.348805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.355617 (+ 6812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.355622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.357533 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.357547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.359301 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.359306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.359479 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.359513 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:33:40.360268 (+ 755us) client_negotiation.cc:240] Negotiation successful
0618 22:33:40.360408 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10610,"thread_start_us":10454,"threads_started":1}
I20260618 22:33:40.361514 26775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828e4658a5564f2da9f603a9a66a105c" candidate_uuid: "62859fc204084afcb527069e201ff03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481d47b030104ed8a169d7c92652eb32"
I20260618 22:33:40.361735 26775 raft_consensus.cc:3037] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:40.361866 27052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828e4658a5564f2da9f603a9a66a105c" candidate_uuid: "481d47b030104ed8a169d7c92652eb32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" is_pre_election: true
I20260618 22:33:40.361915 27053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828e4658a5564f2da9f603a9a66a105c" candidate_uuid: "62859fc204084afcb527069e201ff03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240"
I20260618 22:33:40.362138 27052 raft_consensus.cc:2454] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 481d47b030104ed8a169d7c92652eb32 in term 0.
I20260618 22:33:40.362682 26707 leader_election.cc:302] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 481d47b030104ed8a169d7c92652eb32, d2f1d9712ada46ab9a8d4c5ef8c88240; no voters: 62859fc204084afcb527069e201ff03e
I20260618 22:33:40.363579 27138 leader_election.cc:288] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [CANDIDATE]: Term 1 election: Requested vote from peers 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639), d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131:37527)
I20260618 22:33:40.364502 26775 raft_consensus.cc:2454] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62859fc204084afcb527069e201ff03e in term 1.
I20260618 22:33:40.364634 27133 raft_consensus.cc:2741] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260618 22:33:40.365334 26846 leader_election.cc:302] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 481d47b030104ed8a169d7c92652eb32, 62859fc204084afcb527069e201ff03e; no voters: d2f1d9712ada46ab9a8d4c5ef8c88240
I20260618 22:33:40.365605 27138 raft_consensus.cc:2781] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:40.365765 27138 raft_consensus.cc:686] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 1 LEADER]: Becoming Leader. State: Replica: 62859fc204084afcb527069e201ff03e, State: Running, Role: LEADER
I20260618 22:33:40.366042 27138 consensus_queue.cc:237] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.368561 26616 catalog_manager.cc:5495] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62859fc204084afcb527069e201ff03e (127.18.150.130). New cstate: current_term: 1 leader_uuid: "62859fc204084afcb527069e201ff03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } health_report { overall_health: UNKNOWN } } }
Jun 18 22:33:40 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822018, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.190@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:40.452054 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.436374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33139 (local address 127.0.0.1:40802)
0618 22:33:40.437278 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:40.437293 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:40.437426 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:40.437821 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:40.437828 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:40.438574 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:40.438860 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.438869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.440377 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.440381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.441392 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.441406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.442384 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.442390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.442570 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.443326 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:40.443347 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:40.445779 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:40.449550 (+ 3771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:40.449562 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:40.449568 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:40.450114 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:40.450530 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:40.450533 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:40.450535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:40.450659 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:40.451229 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:40.451235 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:40.451385 (+ 150us) client_negotiation.cc:769] Sending connection context
0618 22:33:40.451513 (+ 128us) client_negotiation.cc:240] Negotiation successful
0618 22:33:40.451715 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":559,"threads_started":1}
I20260618 22:33:40.452383 27166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.436361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40802 (local address 127.18.150.190:33139)
0618 22:33:40.437155 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:40.437162 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:40.437378 (+ 216us) server_negotiation.cc:407] Connection header received
0618 22:33:40.437538 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:40.437543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:40.437625 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:40.437759 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:40.439049 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.440232 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.441563 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.442202 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.442362 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.446034 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:40.446042 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:40.446046 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:40.446087 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:40.449398 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:40.450253 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:40.450258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:40.450262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:40.450424 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:40.450759 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:40.450764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:40.450767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:40.451109 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:40.451259 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:40.451630 (+ 371us) server_negotiation.cc:299] Negotiation successful
0618 22:33:40.451866 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":312,"threads_started":1}
I20260618 22:33:40.457832 26616 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:40802:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260618 22:33:40.458947 26616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:33:40.465860 26755 tablet_service.cc:1467] Processing CreateTablet for tablet f9fb2afbb49c4f89937535f4ca31d8e2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80ef423b789a4c40a56920d1134869c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:33:40.466193 27033 tablet_service.cc:1467] Processing CreateTablet for tablet f9fb2afbb49c4f89937535f4ca31d8e2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80ef423b789a4c40a56920d1134869c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:33:40.466748 26755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9fb2afbb49c4f89937535f4ca31d8e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:40.466756 27033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9fb2afbb49c4f89937535f4ca31d8e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:40.466638 26894 tablet_service.cc:1467] Processing CreateTablet for tablet f9fb2afbb49c4f89937535f4ca31d8e2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=80ef423b789a4c40a56920d1134869c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:33:40.467204 26894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9fb2afbb49c4f89937535f4ca31d8e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:40.473809 27129 tablet_bootstrap.cc:492] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: Bootstrap starting.
I20260618 22:33:40.474583 27128 tablet_bootstrap.cc:492] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e: Bootstrap starting.
I20260618 22:33:40.475524 27129 tablet_bootstrap.cc:654] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:40.476384 27128 tablet_bootstrap.cc:654] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:40.476876 27127 tablet_bootstrap.cc:492] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32: Bootstrap starting.
I20260618 22:33:40.478538 27127 tablet_bootstrap.cc:654] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:40.478624 27128 tablet_bootstrap.cc:492] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e: No bootstrap required, opened a new log
I20260618 22:33:40.478796 27128 ts_tablet_manager.cc:1397] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:33:40.479538 27128 raft_consensus.cc:348] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.479785 27128 raft_consensus.cc:374] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:40.479871 27128 raft_consensus.cc:729] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62859fc204084afcb527069e201ff03e, State: Initialized, Role: FOLLOWER
I20260618 22:33:40.480034 27129 tablet_bootstrap.cc:492] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: No bootstrap required, opened a new log
I20260618 22:33:40.480187 27129 ts_tablet_manager.cc:1397] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260618 22:33:40.480106 27128 consensus_queue.cc:260] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.480676 27128 ts_tablet_manager.cc:1428] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 22:33:40.480561 27138 tablet_replica.cc:439] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } } }
I20260618 22:33:40.480818 27138 tablet_replica.cc:442] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:33:40.480901 27129 raft_consensus.cc:348] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.481072 27129 raft_consensus.cc:374] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:40.481117 27129 raft_consensus.cc:729] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f1d9712ada46ab9a8d4c5ef8c88240, State: Initialized, Role: FOLLOWER
I20260618 22:33:40.481328 27129 consensus_queue.cc:260] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.481698 27134 tablet_replica.cc:439] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } } }
I20260618 22:33:40.482002 27134 tablet_replica.cc:442] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:33:40.483074 27129 ts_tablet_manager.cc:1428] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260618 22:33:40.484611 27127 tablet_bootstrap.cc:492] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32: No bootstrap required, opened a new log
I20260618 22:33:40.484779 27127 ts_tablet_manager.cc:1397] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260618 22:33:40.485335 27127 raft_consensus.cc:348] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.485527 27127 raft_consensus.cc:374] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:40.485599 27127 raft_consensus.cc:729] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 481d47b030104ed8a169d7c92652eb32, State: Initialized, Role: FOLLOWER
I20260618 22:33:40.485868 27127 consensus_queue.cc:260] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.486302 27133 tablet_replica.cc:439] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } } }
I20260618 22:33:40.486552 27133 tablet_replica.cc:442] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:33:40.486967 27127 ts_tablet_manager.cc:1428] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:33:40.666222 27134 raft_consensus.cc:482] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:33:40.666421 27134 raft_consensus.cc:504] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.667833 27134 leader_election.cc:288] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639), 62859fc204084afcb527069e201ff03e (127.18.150.130:41033)
I20260618 22:33:40.673833 27173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.668256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:55259 (local address 127.18.150.129:37639)
0618 22:33:40.669141 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:40.669147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:40.669164 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:33:40.669222 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:40.669225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:40.669278 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:40.669365 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:40.669857 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.670597 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.672206 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.672967 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.673114 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.673214 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:40.673430 (+ 216us) server_negotiation.cc:299] Negotiation successful
0618 22:33:40.673560 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":686,"threads_started":1}
I20260618 22:33:40.673883 27171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.668100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:37639 (local address 127.18.150.131:55259)
0618 22:33:40.668929 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:40.668946 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:40.669056 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:40.669400 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:40.669405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:40.669427 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:40.669699 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.669707 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.670721 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.670725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.672071 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.672085 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.673114 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.673121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.673278 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.673309 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:33:40.673389 (+ 80us) client_negotiation.cc:240] Negotiation successful
0618 22:33:40.673526 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":46,"thread_start_us":269,"threads_started":1}
I20260618 22:33:40.674229 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.668274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:60687 (local address 127.18.150.130:41033)
0618 22:33:40.668533 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:40.668539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:40.668752 (+ 213us) server_negotiation.cc:407] Connection header received
0618 22:33:40.669044 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:40.669049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:40.669188 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:40.669252 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:33:40.669978 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.670658 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.672329 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.673123 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.673300 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.673371 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:40.673635 (+ 264us) server_negotiation.cc:299] Negotiation successful
0618 22:33:40.673763 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260618 22:33:40.674398 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.668100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:41033 (local address 127.18.150.131:60687)
0618 22:33:40.668630 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:40.668654 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:40.668880 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:40.669436 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:40.669439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:40.669459 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:33:40.669794 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.669802 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.670782 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.670786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.672180 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.672192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.673336 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.673343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.673504 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.673526 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:33:40.673599 (+ 73us) client_negotiation.cc:240] Negotiation successful
0618 22:33:40.673789 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":1,"thread_start_us":593,"threads_started":1}
I20260618 22:33:40.674427 26775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fb2afbb49c4f89937535f4ca31d8e2" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481d47b030104ed8a169d7c92652eb32" is_pre_election: true
I20260618 22:33:40.674680 26775 raft_consensus.cc:2454] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 0.
I20260618 22:33:40.674873 26918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fb2afbb49c4f89937535f4ca31d8e2" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62859fc204084afcb527069e201ff03e" is_pre_election: true
I20260618 22:33:40.675076 26918 raft_consensus.cc:2454] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 0.
I20260618 22:33:40.675108 26985 leader_election.cc:302] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 481d47b030104ed8a169d7c92652eb32, d2f1d9712ada46ab9a8d4c5ef8c88240; no voters:
I20260618 22:33:40.675509 27134 raft_consensus.cc:2781] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:33:40.675616 27134 raft_consensus.cc:482] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:33:40.675705 27134 raft_consensus.cc:3037] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:40.677685 27134 raft_consensus.cc:504] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.678228 27134 leader_election.cc:288] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 election: Requested vote from peers 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639), 62859fc204084afcb527069e201ff03e (127.18.150.130:41033)
I20260618 22:33:40.678678 26775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fb2afbb49c4f89937535f4ca31d8e2" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481d47b030104ed8a169d7c92652eb32"
I20260618 22:33:40.678718 26918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9fb2afbb49c4f89937535f4ca31d8e2" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62859fc204084afcb527069e201ff03e"
I20260618 22:33:40.678890 26918 raft_consensus.cc:3037] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:40.678907 26775 raft_consensus.cc:3037] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:40.681720 26775 raft_consensus.cc:2454] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 1.
I20260618 22:33:40.681710 26918 raft_consensus.cc:2454] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 1.
I20260618 22:33:40.682185 26987 leader_election.cc:302] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62859fc204084afcb527069e201ff03e, d2f1d9712ada46ab9a8d4c5ef8c88240; no voters:
I20260618 22:33:40.682495 27134 raft_consensus.cc:2781] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:40.682835 27134 raft_consensus.cc:686] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 LEADER]: Becoming Leader. State: Replica: d2f1d9712ada46ab9a8d4c5ef8c88240, State: Running, Role: LEADER
I20260618 22:33:40.683231 27134 consensus_queue.cc:237] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:40.685101 27174 tablet_replica.cc:439] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: TxnStatusTablet state changed. Reason: New leader d2f1d9712ada46ab9a8d4c5ef8c88240. Latest consensus state: current_term: 1 leader_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } } }
I20260618 22:33:40.685477 27174 tablet_replica.cc:442] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:33:40.686192 27176 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260618 22:33:40.686396 27176 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260618 22:33:40.686872 26616 catalog_manager.cc:5495] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131). New cstate: current_term: 1 leader_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } health_report { overall_health: HEALTHY } } }
I20260618 22:33:40.723387 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.717716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:37527 (local address 127.0.0.1:42062)
0618 22:33:40.717908 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:40.717928 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:40.718088 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:40.718539 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:40.718543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:40.718555 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:33:40.718718 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.718734 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.719750 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.719755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.720894 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.720906 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.721649 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.721655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.721823 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.721879 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:33:40.722872 (+ 993us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:33:40.722888 (+ 16us) client_negotiation.cc:769] Sending connection context
0618 22:33:40.722955 (+ 67us) client_negotiation.cc:240] Negotiation successful
0618 22:33:40.723109 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260618 22:33:40.723398 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.717813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42062 (local address 127.18.150.131:37527)
0618 22:33:40.718029 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:40.718034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:40.718058 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:33:40.718301 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:40.718305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:40.718380 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:40.718498 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:33:40.718848 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.719621 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.721028 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.721504 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.721612 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.722084 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:33:40.722708 (+ 624us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:33:40.722828 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:40.722996 (+ 168us) server_negotiation.cc:299] Negotiation successful
0618 22:33:40.723080 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260618 22:33:40.728241 26775 raft_consensus.cc:1261] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Refusing update from remote peer d2f1d9712ada46ab9a8d4c5ef8c88240: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:40.728437 26918 raft_consensus.cc:1261] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Refusing update from remote peer d2f1d9712ada46ab9a8d4c5ef8c88240: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:40.729212 27134 consensus_queue.cc:1035] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:40.729496 27174 consensus_queue.cc:1035] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:40.732824 27138 tablet_replica.cc:439] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e: TxnStatusTablet state changed. Reason: New leader d2f1d9712ada46ab9a8d4c5ef8c88240. Latest consensus state: current_term: 1 leader_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } } }
I20260618 22:33:40.733064 27138 tablet_replica.cc:442] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:33:40.735991 27134 tablet_replica.cc:439] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } } }
I20260618 22:33:40.736268 27134 tablet_replica.cc:442] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:33:40.736897 27133 tablet_replica.cc:439] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32: TxnStatusTablet state changed. Reason: New leader d2f1d9712ada46ab9a8d4c5ef8c88240. Latest consensus state: current_term: 1 leader_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } } }
I20260618 22:33:40.737164 27133 tablet_replica.cc:442] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:33:40.738168 27179 mvcc.cc:205] Tried to move back new op lower bound from 7298342996895547392 to 7298342996722208768. Current Snapshot: MvccSnapshot[applied={T|T < 7298342996895547392 or (T in {7298342996895547392})}]
I20260618 22:33:40.739526 27174 tablet_replica.cc:439] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } } }
I20260618 22:33:40.739748 27174 tablet_replica.cc:442] T f9fb2afbb49c4f89937535f4ca31d8e2 P d2f1d9712ada46ab9a8d4c5ef8c88240: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:33:40.749063 27138 tablet_replica.cc:439] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } } }
I20260618 22:33:40.749331 27138 tablet_replica.cc:442] T f9fb2afbb49c4f89937535f4ca31d8e2 P 62859fc204084afcb527069e201ff03e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:33:40.751681 27138 consensus_queue.cc:1035] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:40.760228 27133 tablet_replica.cc:439] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } } }
I20260618 22:33:40.760529 27133 tablet_replica.cc:442] T f9fb2afbb49c4f89937535f4ca31d8e2 P 481d47b030104ed8a169d7c92652eb32: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:33:40.764099 27052 raft_consensus.cc:3037] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:40.768931 27138 consensus_queue.cc:1035] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:40.787334 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.780150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:59857 (local address 127.18.150.131:37527)
0618 22:33:40.780330 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:40.780337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:40.781848 (+ 1511us) server_negotiation.cc:407] Connection header received
0618 22:33:40.782150 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:40.782155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:40.782226 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:40.782322 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:33:40.782793 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.783638 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.785600 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.786158 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.786258 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.786746 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:33:40.786849 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:33:40.786915 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:40.787034 (+ 119us) server_negotiation.cc:299] Negotiation successful
0618 22:33:40.787092 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:33:40.787533 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.779894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:37527 (local address 127.18.150.130:59857)
0618 22:33:40.781742 (+ 1848us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:40.781762 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:40.781986 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:40.782410 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:40.782416 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:40.782428 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:33:40.782656 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.782667 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.783807 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.783813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.785454 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.785469 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.786297 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.786304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.786485 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.786564 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:33:40.786934 (+ 370us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:33:40.786945 (+ 11us) client_negotiation.cc:769] Sending connection context
0618 22:33:40.787023 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:33:40.787167 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1667,"thread_start_us":1529,"threads_started":1}
I20260618 22:33:40.829523 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.818648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:57133 (local address 127.18.150.130:41033)
0618 22:33:40.818834 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:40.818841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:40.819303 (+ 462us) server_negotiation.cc:407] Connection header received
0618 22:33:40.819529 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:40.819534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:40.819616 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:40.819678 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:33:40.820738 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.821761 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.825493 (+ 3732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.826193 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.826301 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.828746 (+ 2445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:33:40.828890 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:33:40.828946 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:40.829132 (+ 186us) server_negotiation.cc:299] Negotiation successful
0618 22:33:40.829233 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:33:40.830323 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.818570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:41033 (local address 127.18.150.131:57133)
0618 22:33:40.819196 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:40.819215 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:40.819365 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:40.820103 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:40.820108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:40.820122 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:33:40.820549 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.820562 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.823505 (+ 2943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.823511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.825330 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:40.825346 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.828257 (+ 2911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.828264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:40.828468 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.828540 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:33:40.829016 (+ 476us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:33:40.829027 (+ 11us) client_negotiation.cc:769] Sending connection context
0618 22:33:40.829717 (+ 690us) client_negotiation.cc:240] Negotiation successful
0618 22:33:40.829885 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":970,"threads_started":1}
I20260618 22:33:40.894134 26618 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:40.894362 26618 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:40.897080 26618 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48: Sending DeleteTablet for 3 replicas of tablet 828e4658a5564f2da9f603a9a66a105c
I20260618 22:33:40.897850 26755 tablet_service.cc:1514] Processing DeleteTablet for tablet 828e4658a5564f2da9f603a9a66a105c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:40 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:57678
I20260618 22:33:40.897928 26894 tablet_service.cc:1514] Processing DeleteTablet for tablet 828e4658a5564f2da9f603a9a66a105c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:40 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:44714
I20260618 22:33:40.898061 27033 tablet_service.cc:1514] Processing DeleteTablet for tablet 828e4658a5564f2da9f603a9a66a105c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:40 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:42052
I20260618 22:33:40.908998 27200 tablet_replica.cc:330] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240: stopping tablet replica
I20260618 22:33:40.909273 27200 raft_consensus.cc:2229] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:40.909598 27200 raft_consensus.cc:2258] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:40.910965 27200 ts_tablet_manager.cc:1905] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:40.912822 27199 tablet_replica.cc:330] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e: stopping tablet replica
I20260618 22:33:40.913074 27199 raft_consensus.cc:2229] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:33:40.913256 27198 tablet_replica.cc:330] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32: stopping tablet replica
I20260618 22:33:40.913442 27198 raft_consensus.cc:2229] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:40.913550 27199 raft_consensus.cc:2258] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:40.913648 27198 raft_consensus.cc:2258] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:40.914772 27199 ts_tablet_manager.cc:1905] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:40.914774 27198 ts_tablet_manager.cc:1905] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:40.922701 27199 ts_tablet_manager.cc:1918] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:33:40.922816 27200 ts_tablet_manager.cc:1918] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:33:40.922869 27199 log.cc:1196] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/828e4658a5564f2da9f603a9a66a105c
I20260618 22:33:40.922945 27200 log.cc:1196] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/828e4658a5564f2da9f603a9a66a105c
I20260618 22:33:40.923403 27200 ts_tablet_manager.cc:1939] T 828e4658a5564f2da9f603a9a66a105c P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting consensus metadata
I20260618 22:33:40.923411 27199 ts_tablet_manager.cc:1939] T 828e4658a5564f2da9f603a9a66a105c P 62859fc204084afcb527069e201ff03e: Deleting consensus metadata
I20260618 22:33:40.924731 26604 catalog_manager.cc:4841] TS d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131:37527): tablet 828e4658a5564f2da9f603a9a66a105c (table test-table [id=b0a326b31ab6409e9021910facf99239]) successfully deleted
I20260618 22:33:40.925033 26605 catalog_manager.cc:4841] TS 62859fc204084afcb527069e201ff03e (127.18.150.130:41033): tablet 828e4658a5564f2da9f603a9a66a105c (table test-table [id=b0a326b31ab6409e9021910facf99239]) successfully deleted
I20260618 22:33:40.928604 27198 ts_tablet_manager.cc:1918] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:33:40.928750 27198 log.cc:1196] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/828e4658a5564f2da9f603a9a66a105c
I20260618 22:33:40.929157 27198 ts_tablet_manager.cc:1939] T 828e4658a5564f2da9f603a9a66a105c P 481d47b030104ed8a169d7c92652eb32: Deleting consensus metadata
I20260618 22:33:40.930204 26603 catalog_manager.cc:4841] TS 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639): tablet 828e4658a5564f2da9f603a9a66a105c (table test-table [id=b0a326b31ab6409e9021910facf99239]) successfully deleted
Jun 18 22:33:40 dist-test-slave-h7jz krb5kdc[26573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:33:40 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:40 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:33:40.975826 27208 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:40.958364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40806 (local address 127.18.150.190:33139)
0618 22:33:40.959211 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:40.959216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:40.959237 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:33:40.959336 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:40.959340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:40.959415 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:40.959513 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:40.961400 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.962226 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.964221 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.964720 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.964852 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.975520 (+ 10668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40806: BlockingRecv error: recv got EOF from 127.0.0.1:40806 (error 108)
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":635,"threads_started":1}
Jun 18 22:33:40 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM
I20260618 22:33:40.993116 27208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:40.978595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40808 (local address 127.18.150.190:33139)
0618 22:33:40.978783 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:40.978789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:40.979095 (+ 306us) server_negotiation.cc:407] Connection header received
0618 22:33:40.979296 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:40.979300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:40.979366 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:40.979447 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:40.980858 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.981705 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.983500 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:40.983949 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:40.984067 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:40.988148 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:40.988153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:40.988157 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:40.988186 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:40.990624 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:40.991359 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:40.991363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:40.991365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:40.991492 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:40.991925 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:40.991930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:40.991933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:40.992198 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:40.992384 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:40.992699 (+ 315us) server_negotiation.cc:299] Negotiation successful
0618 22:33:40.992872 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:33:40.996483 26618 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:33:40.997066 26618 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:33:41.003988 26755 tablet_service.cc:1467] Processing CreateTablet for tablet 57c4b72733444d4da7fe738ac0546fa1 (DEFAULT_TABLE table=test-table [id=01406c2962974917984f9c963976e48a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:41.004022 26894 tablet_service.cc:1467] Processing CreateTablet for tablet 57c4b72733444d4da7fe738ac0546fa1 (DEFAULT_TABLE table=test-table [id=01406c2962974917984f9c963976e48a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:41.004475 26755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c4b72733444d4da7fe738ac0546fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:41.004568 26894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c4b72733444d4da7fe738ac0546fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:41.009605 27033 tablet_service.cc:1467] Processing CreateTablet for tablet 57c4b72733444d4da7fe738ac0546fa1 (DEFAULT_TABLE table=test-table [id=01406c2962974917984f9c963976e48a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:41.010102 27033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c4b72733444d4da7fe738ac0546fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:41.010627 27214 tablet_bootstrap.cc:492] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32: Bootstrap starting.
I20260618 22:33:41.012220 27214 tablet_bootstrap.cc:654] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:41.019385 27215 tablet_bootstrap.cc:492] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e: Bootstrap starting.
I20260618 22:33:41.020893 27217 tablet_bootstrap.cc:492] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240: Bootstrap starting.
I20260618 22:33:41.022203 27215 tablet_bootstrap.cc:654] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:41.022871 27217 tablet_bootstrap.cc:654] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:41.024708 27214 tablet_bootstrap.cc:492] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32: No bootstrap required, opened a new log
I20260618 22:33:41.024884 27214 ts_tablet_manager.cc:1397] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.007s
I20260618 22:33:41.025701 27214 raft_consensus.cc:348] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.026042 27214 raft_consensus.cc:374] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:41.026139 27214 raft_consensus.cc:729] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 481d47b030104ed8a169d7c92652eb32, State: Initialized, Role: FOLLOWER
I20260618 22:33:41.026409 27214 consensus_queue.cc:260] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.027662 27214 ts_tablet_manager.cc:1428] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:41.033499 27215 tablet_bootstrap.cc:492] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e: No bootstrap required, opened a new log
I20260618 22:33:41.033651 27215 ts_tablet_manager.cc:1397] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260618 22:33:41.033970 27217 tablet_bootstrap.cc:492] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240: No bootstrap required, opened a new log
I20260618 22:33:41.034154 27217 ts_tablet_manager.cc:1397] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.005s
I20260618 22:33:41.034401 27215 raft_consensus.cc:348] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.034595 27215 raft_consensus.cc:374] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:41.034675 27215 raft_consensus.cc:729] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62859fc204084afcb527069e201ff03e, State: Initialized, Role: FOLLOWER
I20260618 22:33:41.034945 27215 consensus_queue.cc:260] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.035193 27217 raft_consensus.cc:348] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.035475 27217 raft_consensus.cc:374] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:41.035543 27215 ts_tablet_manager.cc:1428] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:33:41.035581 27217 raft_consensus.cc:729] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f1d9712ada46ab9a8d4c5ef8c88240, State: Initialized, Role: FOLLOWER
I20260618 22:33:41.035900 27217 consensus_queue.cc:260] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.036542 27217 ts_tablet_manager.cc:1428] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260618 22:33:41.081957 27194 raft_consensus.cc:482] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:33:41.082175 27194 raft_consensus.cc:504] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.082978 27194 leader_election.cc:288] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639), 62859fc204084afcb527069e201ff03e (127.18.150.130:41033)
I20260618 22:33:41.083604 26918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c4b72733444d4da7fe738ac0546fa1" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62859fc204084afcb527069e201ff03e" is_pre_election: true
I20260618 22:33:41.083604 26774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c4b72733444d4da7fe738ac0546fa1" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481d47b030104ed8a169d7c92652eb32" is_pre_election: true
I20260618 22:33:41.083851 26918 raft_consensus.cc:2454] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 0.
I20260618 22:33:41.083849 26774 raft_consensus.cc:2454] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 0.
I20260618 22:33:41.084313 26985 leader_election.cc:302] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 481d47b030104ed8a169d7c92652eb32, d2f1d9712ada46ab9a8d4c5ef8c88240; no voters:
I20260618 22:33:41.084619 27194 raft_consensus.cc:2781] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:33:41.084764 27194 raft_consensus.cc:482] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:33:41.084867 27194 raft_consensus.cc:3037] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:41.088130 27194 raft_consensus.cc:504] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.088907 27194 leader_election.cc:288] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 election: Requested vote from peers 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639), 62859fc204084afcb527069e201ff03e (127.18.150.130:41033)
I20260618 22:33:41.089360 26918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c4b72733444d4da7fe738ac0546fa1" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62859fc204084afcb527069e201ff03e"
I20260618 22:33:41.089349 26774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c4b72733444d4da7fe738ac0546fa1" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481d47b030104ed8a169d7c92652eb32"
I20260618 22:33:41.089547 26918 raft_consensus.cc:3037] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:41.089568 26774 raft_consensus.cc:3037] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:41.091596 26918 raft_consensus.cc:2454] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 1.
I20260618 22:33:41.091962 26987 leader_election.cc:302] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62859fc204084afcb527069e201ff03e, d2f1d9712ada46ab9a8d4c5ef8c88240; no voters:
I20260618 22:33:41.092222 27194 raft_consensus.cc:2781] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:41.092355 27194 raft_consensus.cc:686] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 LEADER]: Becoming Leader. State: Replica: d2f1d9712ada46ab9a8d4c5ef8c88240, State: Running, Role: LEADER
I20260618 22:33:41.092629 27194 consensus_queue.cc:237] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.092769 26774 raft_consensus.cc:2454] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 1.
I20260618 22:33:41.095995 26618 catalog_manager.cc:5495] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131). New cstate: current_term: 1 leader_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } health_report { overall_health: HEALTHY } } }
I20260618 22:33:41.115931 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:41.110395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42064 (local address 127.18.150.131:37527)
0618 22:33:41.110537 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:41.110542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:41.110827 (+ 285us) server_negotiation.cc:407] Connection header received
0618 22:33:41.111031 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:41.111034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:41.111096 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:41.111165 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:33:41.111552 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.112325 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.114302 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.114734 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.114829 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:41.115355 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:33:41.115453 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:33:41.115509 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:41.115642 (+ 133us) server_negotiation.cc:299] Negotiation successful
0618 22:33:41.115704 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:33:41.119139 26774 raft_consensus.cc:1261] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Refusing update from remote peer d2f1d9712ada46ab9a8d4c5ef8c88240: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:41.119146 26918 raft_consensus.cc:1261] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Refusing update from remote peer d2f1d9712ada46ab9a8d4c5ef8c88240: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:41.119792 27134 consensus_queue.cc:1035] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:41.120119 27194 consensus_queue.cc:1035] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:41.135669 27180 mvcc.cc:205] Tried to move back new op lower bound from 7298342998497984512 to 7298342998399184896. Current Snapshot: MvccSnapshot[applied={T|T < 7298342998497984512}]
I20260618 22:33:41.136291 26616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:41.137115 26616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:41.139958 26616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48: Sending DeleteTablet for 3 replicas of tablet 57c4b72733444d4da7fe738ac0546fa1
I20260618 22:33:41.140658 26755 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c4b72733444d4da7fe738ac0546fa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:41 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:57678
I20260618 22:33:41.140916 27198 tablet_replica.cc:330] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32: stopping tablet replica
I20260618 22:33:41.140954 27033 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c4b72733444d4da7fe738ac0546fa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:41 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:42052
I20260618 22:33:41.141114 27198 raft_consensus.cc:2229] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:41.141222 27200 tablet_replica.cc:330] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240: stopping tablet replica
I20260618 22:33:41.141325 27198 raft_consensus.cc:2258] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:41.141448 27200 raft_consensus.cc:2229] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:33:41.141710 26894 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c4b72733444d4da7fe738ac0546fa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:41 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:44714
I20260618 22:33:41.141985 27200 raft_consensus.cc:2258] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:41.142055 27199 tablet_replica.cc:330] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e: stopping tablet replica
I20260618 22:33:41.142222 27199 raft_consensus.cc:2229] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:41.142427 27199 raft_consensus.cc:2258] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:41.143134 27198 ts_tablet_manager.cc:1905] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:41.143867 26616 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:33:41.144481 26616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:33:41.146317 27200 ts_tablet_manager.cc:1905] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:41.147624 27199 ts_tablet_manager.cc:1905] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:41.152109 26894 tablet_service.cc:1467] Processing CreateTablet for tablet 9143ab501c0347a7a15457febde8269d (DEFAULT_TABLE table=test-table [id=9d377802ddfd43f4bf36a0881467b9ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:41.152129 27033 tablet_service.cc:1467] Processing CreateTablet for tablet 9143ab501c0347a7a15457febde8269d (DEFAULT_TABLE table=test-table [id=9d377802ddfd43f4bf36a0881467b9ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:41.152124 26755 tablet_service.cc:1467] Processing CreateTablet for tablet 9143ab501c0347a7a15457febde8269d (DEFAULT_TABLE table=test-table [id=9d377802ddfd43f4bf36a0881467b9ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:33:41.152505 26755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9143ab501c0347a7a15457febde8269d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:41.152508 27033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9143ab501c0347a7a15457febde8269d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:41.152518 26894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9143ab501c0347a7a15457febde8269d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:41.161011 27199 ts_tablet_manager.cc:1918] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:41.161233 27199 log.cc:1196] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/57c4b72733444d4da7fe738ac0546fa1
I20260618 22:33:41.161628 27199 ts_tablet_manager.cc:1939] T 57c4b72733444d4da7fe738ac0546fa1 P 62859fc204084afcb527069e201ff03e: Deleting consensus metadata
I20260618 22:33:41.161809 27214 tablet_bootstrap.cc:492] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32: Bootstrap starting.
I20260618 22:33:41.162644 26605 catalog_manager.cc:4841] TS 62859fc204084afcb527069e201ff03e (127.18.150.130:41033): tablet 57c4b72733444d4da7fe738ac0546fa1 (table test-table [id=01406c2962974917984f9c963976e48a]) successfully deleted
I20260618 22:33:41.163129 27215 tablet_bootstrap.cc:492] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e: Bootstrap starting.
I20260618 22:33:41.163102 27200 ts_tablet_manager.cc:1918] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:41.163250 27200 log.cc:1196] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/57c4b72733444d4da7fe738ac0546fa1
I20260618 22:33:41.163375 27214 tablet_bootstrap.cc:654] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:41.163684 27200 ts_tablet_manager.cc:1939] T 57c4b72733444d4da7fe738ac0546fa1 P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting consensus metadata
I20260618 22:33:41.164602 27215 tablet_bootstrap.cc:654] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:41.165510 27217 tablet_bootstrap.cc:492] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240: Bootstrap starting.
I20260618 22:33:41.166404 27198 ts_tablet_manager.cc:1918] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:33:41.166694 27198 log.cc:1196] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/57c4b72733444d4da7fe738ac0546fa1
I20260618 22:33:41.167273 27198 ts_tablet_manager.cc:1939] T 57c4b72733444d4da7fe738ac0546fa1 P 481d47b030104ed8a169d7c92652eb32: Deleting consensus metadata
I20260618 22:33:41.168721 26603 catalog_manager.cc:4841] TS 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639): tablet 57c4b72733444d4da7fe738ac0546fa1 (table test-table [id=01406c2962974917984f9c963976e48a]) successfully deleted
I20260618 22:33:41.169486 26604 catalog_manager.cc:4841] TS d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131:37527): tablet 57c4b72733444d4da7fe738ac0546fa1 (table test-table [id=01406c2962974917984f9c963976e48a]) successfully deleted
I20260618 22:33:41.169971 27217 tablet_bootstrap.cc:654] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:41.173243 27215 tablet_bootstrap.cc:492] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e: No bootstrap required, opened a new log
I20260618 22:33:41.173447 27215 ts_tablet_manager.cc:1397] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.008s
I20260618 22:33:41.174515 27214 tablet_bootstrap.cc:492] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32: No bootstrap required, opened a new log
I20260618 22:33:41.174479 27215 raft_consensus.cc:348] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.174737 27215 raft_consensus.cc:374] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:41.174811 27215 raft_consensus.cc:729] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62859fc204084afcb527069e201ff03e, State: Initialized, Role: FOLLOWER
I20260618 22:33:41.175148 27214 ts_tablet_manager.cc:1397] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.001s
I20260618 22:33:41.175062 27215 consensus_queue.cc:260] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.175668 27215 ts_tablet_manager.cc:1428] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:33:41.176023 27214 raft_consensus.cc:348] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.176337 27214 raft_consensus.cc:374] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:41.176455 27214 raft_consensus.cc:729] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 481d47b030104ed8a169d7c92652eb32, State: Initialized, Role: FOLLOWER
I20260618 22:33:41.176735 27214 consensus_queue.cc:260] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.177349 27214 ts_tablet_manager.cc:1428] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260618 22:33:41.179425 27217 tablet_bootstrap.cc:492] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240: No bootstrap required, opened a new log
I20260618 22:33:41.179575 27217 ts_tablet_manager.cc:1397] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.005s
I20260618 22:33:41.180181 27217 raft_consensus.cc:348] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.180428 27217 raft_consensus.cc:374] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:41.180526 27217 raft_consensus.cc:729] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f1d9712ada46ab9a8d4c5ef8c88240, State: Initialized, Role: FOLLOWER
I20260618 22:33:41.180769 27217 consensus_queue.cc:260] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.181308 27217 ts_tablet_manager.cc:1428] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260618 22:33:41.197074 27194 raft_consensus.cc:482] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:33:41.197244 27194 raft_consensus.cc:504] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.197818 27194 leader_election.cc:288] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639), 62859fc204084afcb527069e201ff03e (127.18.150.130:41033)
I20260618 22:33:41.198304 26774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9143ab501c0347a7a15457febde8269d" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481d47b030104ed8a169d7c92652eb32" is_pre_election: true
I20260618 22:33:41.198377 26918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9143ab501c0347a7a15457febde8269d" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62859fc204084afcb527069e201ff03e" is_pre_election: true
I20260618 22:33:41.198508 26774 raft_consensus.cc:2454] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 0.
I20260618 22:33:41.198624 26918 raft_consensus.cc:2454] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 0.
I20260618 22:33:41.198958 26985 leader_election.cc:302] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 481d47b030104ed8a169d7c92652eb32, d2f1d9712ada46ab9a8d4c5ef8c88240; no voters:
I20260618 22:33:41.199208 27194 raft_consensus.cc:2781] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:33:41.199319 27194 raft_consensus.cc:482] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:33:41.199407 27194 raft_consensus.cc:3037] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:41.201449 27194 raft_consensus.cc:504] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.202025 27194 leader_election.cc:288] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 election: Requested vote from peers 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639), 62859fc204084afcb527069e201ff03e (127.18.150.130:41033)
I20260618 22:33:41.202445 26774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9143ab501c0347a7a15457febde8269d" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481d47b030104ed8a169d7c92652eb32"
I20260618 22:33:41.202651 26774 raft_consensus.cc:3037] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:41.202808 26918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9143ab501c0347a7a15457febde8269d" candidate_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62859fc204084afcb527069e201ff03e"
I20260618 22:33:41.203006 26918 raft_consensus.cc:3037] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:41.205010 26774 raft_consensus.cc:2454] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 1.
I20260618 22:33:41.205359 26985 leader_election.cc:302] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 481d47b030104ed8a169d7c92652eb32, d2f1d9712ada46ab9a8d4c5ef8c88240; no voters:
I20260618 22:33:41.205641 27194 raft_consensus.cc:2781] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:41.205823 27194 raft_consensus.cc:686] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 LEADER]: Becoming Leader. State: Replica: d2f1d9712ada46ab9a8d4c5ef8c88240, State: Running, Role: LEADER
I20260618 22:33:41.206089 26918 raft_consensus.cc:2454] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f1d9712ada46ab9a8d4c5ef8c88240 in term 1.
I20260618 22:33:41.206118 27194 consensus_queue.cc:237] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } }
I20260618 22:33:41.208634 26616 catalog_manager.cc:5495] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131). New cstate: current_term: 1 leader_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2f1d9712ada46ab9a8d4c5ef8c88240" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37527 } health_report { overall_health: HEALTHY } } }
I20260618 22:33:41.236661 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:41.232129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:37527 (local address 127.18.150.131:38183)
0618 22:33:41.232281 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:41.232305 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:41.232419 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:41.232697 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:41.232702 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:41.232711 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:33:41.232875 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:41.232883 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.233822 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.233827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:41.234908 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:41.234920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.235619 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.235625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:41.235770 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:41.235808 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:33:41.236123 (+ 315us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:33:41.236133 (+ 10us) client_negotiation.cc:769] Sending connection context
0618 22:33:41.236203 (+ 70us) client_negotiation.cc:240] Negotiation successful
0618 22:33:41.236334 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260618 22:33:41.236670 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:41.232064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:38183 (local address 127.18.150.131:37527)
0618 22:33:41.232245 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:41.232251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:41.232363 (+ 112us) server_negotiation.cc:407] Connection header received
0618 22:33:41.232545 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:41.232548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:41.232606 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:41.232672 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:33:41.232981 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.233700 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.235032 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.235485 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.235594 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:41.235935 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:33:41.236037 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:33:41.236113 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:41.236248 (+ 135us) server_negotiation.cc:299] Negotiation successful
0618 22:33:41.236326 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:33:41.244313 26918 raft_consensus.cc:1261] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Refusing update from remote peer d2f1d9712ada46ab9a8d4c5ef8c88240: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:41.244421 26774 raft_consensus.cc:1261] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Refusing update from remote peer d2f1d9712ada46ab9a8d4c5ef8c88240: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:33:41.244809 27194 consensus_queue.cc:1035] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62859fc204084afcb527069e201ff03e" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:41.245056 27134 consensus_queue.cc:1035] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [LEADER]: Connected to new peer: Peer: permanent_uuid: "481d47b030104ed8a169d7c92652eb32" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:33:41.294198 26616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:41.294442 26616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:33:41.297871 26616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29728dc62c584cce9d03adf34ec39f48: Sending DeleteTablet for 3 replicas of tablet 9143ab501c0347a7a15457febde8269d
I20260618 22:33:41.298628 26755 tablet_service.cc:1514] Processing DeleteTablet for tablet 9143ab501c0347a7a15457febde8269d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:41 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:57678
I20260618 22:33:41.298794 27032 tablet_service.cc:1514] Processing DeleteTablet for tablet 9143ab501c0347a7a15457febde8269d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:41 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:42052
I20260618 22:33:41.298918 27198 tablet_replica.cc:330] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32: stopping tablet replica
I20260618 22:33:41.299080 27198 raft_consensus.cc:2229] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:41.299115 27200 tablet_replica.cc:330] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240: stopping tablet replica
I20260618 22:33:41.299278 27198 raft_consensus.cc:2258] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:41.299294 27200 raft_consensus.cc:2229] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:33:41.299607 27200 raft_consensus.cc:2258] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:41.299623 26894 tablet_service.cc:1514] Processing DeleteTablet for tablet 9143ab501c0347a7a15457febde8269d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:33:41 UTC) from {username='oryx', principal='oryx/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:44714
I20260618 22:33:41.299988 27199 tablet_replica.cc:330] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e: stopping tablet replica
I20260618 22:33:41.300166 27199 raft_consensus.cc:2229] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:33:41.300237 27198 ts_tablet_manager.cc:1905] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:41.300338 27199 raft_consensus.cc:2258] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:33:41.300645 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 26689
I20260618 22:33:41.301281 27200 ts_tablet_manager.cc:1905] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:41.307024 27198 ts_tablet_manager.cc:1918] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:33:41.307144 27198 log.cc:1196] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/9143ab501c0347a7a15457febde8269d
I20260618 22:33:41.307597 27198 ts_tablet_manager.cc:1939] T 9143ab501c0347a7a15457febde8269d P 481d47b030104ed8a169d7c92652eb32: Deleting consensus metadata
I20260618 22:33:41.308105 27200 ts_tablet_manager.cc:1918] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:33:41.308238 27200 log.cc:1196] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/9143ab501c0347a7a15457febde8269d
I20260618 22:33:41.308574 26603 catalog_manager.cc:4841] TS 481d47b030104ed8a169d7c92652eb32 (127.18.150.129:37639): tablet 9143ab501c0347a7a15457febde8269d (table test-table [id=9d377802ddfd43f4bf36a0881467b9ad]) successfully deleted
I20260618 22:33:41.308642 27200 ts_tablet_manager.cc:1939] T 9143ab501c0347a7a15457febde8269d P d2f1d9712ada46ab9a8d4c5ef8c88240: Deleting consensus metadata
I20260618 22:33:41.309638 26604 catalog_manager.cc:4841] TS d2f1d9712ada46ab9a8d4c5ef8c88240 (127.18.150.131:37527): tablet 9143ab501c0347a7a15457febde8269d (table test-table [id=9d377802ddfd43f4bf36a0881467b9ad]) successfully deleted
I20260618 22:33:41.310878 27199 ts_tablet_manager.cc:1905] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:33:41.316103 27199 ts_tablet_manager.cc:1918] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:33:41.316211 27199 log.cc:1196] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/9143ab501c0347a7a15457febde8269d
I20260618 22:33:41.316480 27199 ts_tablet_manager.cc:1939] T 9143ab501c0347a7a15457febde8269d P 62859fc204084afcb527069e201ff03e: Deleting consensus metadata
I20260618 22:33:41.317452 26605 catalog_manager.cc:4841] TS 62859fc204084afcb527069e201ff03e (127.18.150.130:41033): tablet 9143ab501c0347a7a15457febde8269d (table test-table [id=9d377802ddfd43f4bf36a0881467b9ad]) successfully deleted
Jun 18 22:33:41 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.150.129@KRBTEST.COM
I20260618 22:33:41.333113 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:41.303605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57690 (local address 127.18.150.129:37639)
0618 22:33:41.310304 (+ 6699us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:41.310311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:41.318921 (+ 8610us) server_negotiation.cc:407] Connection header received
0618 22:33:41.319097 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:41.319101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:41.319168 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:41.319275 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:41.320471 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.321150 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.322619 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.323088 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.323194 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:41.327216 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:41.327232 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:41.327248 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:41.327307 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:41.330402 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:41.331165 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:41.331173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:41.331175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:41.331298 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:41.331651 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:41.331655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:41.331657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:41.332010 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:41.332285 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:41.332556 (+ 271us) server_negotiation.cc:299] Negotiation successful
0618 22:33:41.332771 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6588,"thread_start_us":6505,"threads_started":1}
I20260618 22:33:41.333722 26820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:41.429684 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 26689
I20260618 22:33:41.444643 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 26829
Jun 18 22:33:41 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.150.130@KRBTEST.COM
I20260618 22:33:41.461688 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:41.445426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44732 (local address 127.18.150.130:41033)
0618 22:33:41.445843 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:41.445847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:41.445863 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:33:41.445915 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:41.445918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:41.446042 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:41.446164 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:41.447645 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.448361 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.450100 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.450496 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.450601 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:41.455505 (+ 4904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:41.455524 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:41.455546 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:41.455592 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:41.459097 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:41.459727 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:41.459736 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:41.459739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:41.459893 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:41.460193 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:41.460198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:41.460201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:41.460619 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:41.460921 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:41.461065 (+ 144us) server_negotiation.cc:299] Negotiation successful
0618 22:33:41.461310 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":221,"threads_started":1}
I20260618 22:33:41.462356 26959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:41.582134 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 26829
I20260618 22:33:41.600242 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 26968
Jun 18 22:33:41 dist-test-slave-h7jz krb5kdc[26573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.150.131@KRBTEST.COM
I20260618 22:33:41.614637 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:41.600966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.18.150.131:37527)
0618 22:33:41.601140 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:41.601145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:41.601163 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:33:41.601269 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:41.601274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:41.601341 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:41.601423 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:41.602677 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.603404 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.604646 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:41.605061 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:41.605220 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:41.609459 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:41.609473 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:41.609490 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:41.609521 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:41.612045 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:41.612631 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:41.612673 (+ 42us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:41.612677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:41.612813 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:41.613148 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:41.613153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:41.613157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:41.613501 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:41.613768 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:41.614095 (+ 327us) server_negotiation.cc:299] Negotiation successful
0618 22:33:41.614328 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:33:41.615200 27098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:41.760218 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 26968
I20260618 22:33:41.780225 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 26586
I20260618 22:33:41.781352 26658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:41.897081 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 26586
2026-06-18T22:33:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5252 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:33:41 dist-test-slave-h7jz krb5kdc[27241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:33:41 dist-test-slave-h7jz krb5kdc[27241](info): set up 2 sockets
Jun 18 22:33:41 dist-test-slave-h7jz krb5kdc[27241](info): commencing operation
krb5kdc: starting...
W20260618 22:33:42.902817 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.965s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:33:43 dist-test-slave-h7jz krb5kdc[27241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:33:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:33:43.107465 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:32833
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:43411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:32833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:43.415585 27254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:43.415858 27254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:43.415989 27254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:43.423022 27254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:43.423137 27254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:43.423215 27254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:43.423291 27254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:43.423349 27254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:43.432934 27254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:43411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:32833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:32833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:43.434273 27254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:43.435616 27254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:43.447000 27259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:43.447059 27262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:43.447016 27260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:43.447782 27254 server_base.cc:1034] running on GCE node
I20260618 22:33:43.448591 27254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:43.450050 27254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:43.451247 27254 hybrid_clock.cc:648] HybridClock initialized: now 1781822023451190 us; error 48 us; skew 500 ppm
I20260618 22:33:43.451568 27254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-18T22:33:43Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1618 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:33:43 dist-test-slave-h7jz krb5kdc[27268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:33:43 dist-test-slave-h7jz krb5kdc[27268](info): set up 2 sockets
Jun 18 22:33:43 dist-test-slave-h7jz krb5kdc[27268](info): commencing operation
krb5kdc: starting...
W20260618 22:33:44.526643 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:33:44 dist-test-slave-h7jz krb5kdc[27268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:33:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:33:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:33:44.730742 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:42333
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:42333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:45.048684 27281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:45.048997 27281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:45.049125 27281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:45.056398 27281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:45.056524 27281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:45.056577 27281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:45.056659 27281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:45.056726 27281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:45.067077 27281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:42333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:42333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:45.068486 27281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:45.070116 27281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:45.082916 27286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:45.083020 27287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:45.082945 27289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:45.083698 27281 server_base.cc:1034] running on GCE node
I20260618 22:33:45.084648 27281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:45.086411 27281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:45.087687 27281 hybrid_clock.cc:648] HybridClock initialized: now 1781822025087636 us; error 44 us; skew 500 ppm
I20260618 22:33:45.088099 27281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:45 dist-test-slave-h7jz krb5kdc[27268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822025, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:45.091579 27281 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:33:45.093362 27281 webserver.cc:460] Webserver started at http://127.18.150.190:35689/ using document root <none> and password file <none>
I20260618 22:33:45.094244 27281 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:45.094388 27281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:45.094707 27281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:45.098129 27281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "74d1f5798cc942f99ef91d06eda38dff"
format_stamp: "Formatted at 2026-06-18 22:33:45 on dist-test-slave-h7jz"
server_key: "8d5864ff6a70419875fd8c322caca4ab"
server_key_iv: "c5265d576c85b67ba17ea80fcad065ab"
server_key_version: "encryptionkey@0"
I20260618 22:33:45.099109 27281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "74d1f5798cc942f99ef91d06eda38dff"
format_stamp: "Formatted at 2026-06-18 22:33:45 on dist-test-slave-h7jz"
server_key: "8d5864ff6a70419875fd8c322caca4ab"
server_key_iv: "c5265d576c85b67ba17ea80fcad065ab"
server_key_version: "encryptionkey@0"
I20260618 22:33:45.105330 27281 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260618 22:33:45.109546 27296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:45.111256 27281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:45.111493 27281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "74d1f5798cc942f99ef91d06eda38dff"
format_stamp: "Formatted at 2026-06-18 22:33:45 on dist-test-slave-h7jz"
server_key: "8d5864ff6a70419875fd8c322caca4ab"
server_key_iv: "c5265d576c85b67ba17ea80fcad065ab"
server_key_version: "encryptionkey@0"
I20260618 22:33:45.111769 27281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:45.130857 27281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:45.135591 27281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:45.135903 27281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:45.154616 27281 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:42333
I20260618 22:33:45.154659 27347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:42333 every 8 connection(s)
I20260618 22:33:45.155983 27281 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:45.159722 27348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:33:45.160084 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 27281
I20260618 22:33:45.160297 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:33:45.161130 19034 external_mini_cluster.cc:1428] Setting key a7724ed5405a6bb25fd7a61806868e81
I20260618 22:33:45.173539 27348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff: Bootstrap starting.
Jun 18 22:33:45 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:45.177227 27348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff: Neither blocks nor log segments found. Creating new log.
I20260618 22:33:45.178397 27348 log.cc:824] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:45.181484 27348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff: No bootstrap required, opened a new log
I20260618 22:33:45.184226 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:45.163141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48094 (local address 127.18.150.190:42333)
0618 22:33:45.163868 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:45.163879 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:45.163919 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:33:45.165984 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:45.166014 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:45.166429 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:45.166780 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:45.168438 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:45.169710 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:45.171491 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:45.172148 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:45.172274 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:45.176835 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:45.176843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:45.176857 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:45.176892 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:45.180333 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:45.181091 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:45.181097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:45.181102 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:45.181254 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:45.181633 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:45.181637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:45.181640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:45.182194 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:45.182416 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:45.182835 (+ 419us) server_negotiation.cc:299] Negotiation successful
0618 22:33:45.183164 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":319,"threads_started":1}
I20260618 22:33:45.189148 27348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } }
I20260618 22:33:45.189474 27348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:33:45.189555 27348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74d1f5798cc942f99ef91d06eda38dff, State: Initialized, Role: FOLLOWER
I20260618 22:33:45.190096 27348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } }
I20260618 22:33:45.190303 27348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:45.190415 27348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:45.190551 27348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:33:45.192596 27348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } }
I20260618 22:33:45.193037 27348 leader_election.cc:302] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74d1f5798cc942f99ef91d06eda38dff; no voters:
I20260618 22:33:45.193526 27348 leader_election.cc:288] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:33:45.193660 27353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:33:45.194271 27353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 1 LEADER]: Becoming Leader. State: Replica: 74d1f5798cc942f99ef91d06eda38dff, State: Running, Role: LEADER
I20260618 22:33:45.194747 27353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } }
I20260618 22:33:45.195379 27348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:45.197126 27354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74d1f5798cc942f99ef91d06eda38dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } } }
I20260618 22:33:45.197517 27354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:45.197153 27355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74d1f5798cc942f99ef91d06eda38dff. Latest consensus state: current_term: 1 leader_uuid: "74d1f5798cc942f99ef91d06eda38dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } } }
I20260618 22:33:45.199188 27355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:45.200703 27359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:45.205621 27359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:45.213640 27359 catalog_manager.cc:1347] Generated new cluster ID: dfc94adad3754baca973c1b327d326ff
I20260618 22:33:45.213742 27359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:45.229929 27359 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:33:45.231892 27359 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:45.238893 27359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff: Generated new TSK 0
I20260618 22:33:45.239889 27359 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:33:45.296861 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:42333
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:45.610522 27374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:45.610806 27374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:45.610932 27374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:45.618333 27374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:45.618436 27374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:45.618582 27374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:33:45.628226 27374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:42333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:45.629483 27374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:45.630743 27374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:45.643257 27379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:45.643148 27380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:45.643148 27382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:45.643829 27374 server_base.cc:1034] running on GCE node
I20260618 22:33:45.644439 27374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:45.645368 27374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:45.646570 27374 hybrid_clock.cc:648] HybridClock initialized: now 1781822025646521 us; error 41 us; skew 500 ppm
I20260618 22:33:45.646869 27374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:45 dist-test-slave-h7jz krb5kdc[27268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822025, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:45.649832 27374 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:33:45.651201 27374 webserver.cc:460] Webserver started at http://127.18.150.129:35551/ using document root <none> and password file <none>
I20260618 22:33:45.651846 27374 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:45.651942 27374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:45.652181 27374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:45.655030 27374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "4b3079f2f16e4d72a26539c993259cf6"
format_stamp: "Formatted at 2026-06-18 22:33:45 on dist-test-slave-h7jz"
server_key: "13c22d4c1c4b623a99290b3c1f3f1c55"
server_key_iv: "d1b356af4eac04e4d6c7f0dcfa6f7f31"
server_key_version: "encryptionkey@0"
I20260618 22:33:45.655772 27374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "4b3079f2f16e4d72a26539c993259cf6"
format_stamp: "Formatted at 2026-06-18 22:33:45 on dist-test-slave-h7jz"
server_key: "13c22d4c1c4b623a99290b3c1f3f1c55"
server_key_iv: "d1b356af4eac04e4d6c7f0dcfa6f7f31"
server_key_version: "encryptionkey@0"
I20260618 22:33:45.660776 27374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:33:45.664079 27389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:45.665356 27374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:33:45.665485 27374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "4b3079f2f16e4d72a26539c993259cf6"
format_stamp: "Formatted at 2026-06-18 22:33:45 on dist-test-slave-h7jz"
server_key: "13c22d4c1c4b623a99290b3c1f3f1c55"
server_key_iv: "d1b356af4eac04e4d6c7f0dcfa6f7f31"
server_key_version: "encryptionkey@0"
I20260618 22:33:45.665639 27374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:45.702252 27374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:45.706360 27374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:45.706625 27374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:45.707504 27374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:45.708732 27374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:45.708813 27374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:45.708901 27374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:45.708942 27374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:45.738529 27374 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:44067
I20260618 22:33:45.738565 27501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:44067 every 8 connection(s)
I20260618 22:33:45.739778 27374 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:33:45.749708 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 27374
I20260618 22:33:45.749904 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:33:45.750514 19034 external_mini_cluster.cc:1428] Setting key 39e8076636614810b30321163515367f
Jun 18 22:33:45 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822025, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:45.758733 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:45.742220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:49047 (local address 127.18.150.190:42333)
0618 22:33:45.742634 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:45.742638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:45.743504 (+ 866us) server_negotiation.cc:407] Connection header received
0618 22:33:45.745263 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:45.745269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:45.745353 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:45.745448 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:45.747081 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:45.747797 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:45.749251 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:45.749658 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:45.749780 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:45.753067 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:45.753072 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:45.753075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:45.753103 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:45.755682 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:45.756503 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:45.756511 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:45.756514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:45.756629 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:45.757036 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:45.757040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:45.757043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:45.757299 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:45.757419 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:45.758213 (+ 794us) server_negotiation.cc:299] Negotiation successful
0618 22:33:45.758423 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":209,"threads_started":1}
I20260618 22:33:45.760013 27505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:45.742708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:49047)
0618 22:33:45.743357 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:45.743395 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:45.745003 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:45.745684 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:45.745691 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:45.746296 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:45.746918 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:45.746940 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:45.747939 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:45.747943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:45.749121 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:45.749132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:45.749865 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:45.749872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:45.750138 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:45.750741 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:45.750762 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:45.752880 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:45.755817 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:45.755827 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:45.755843 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:45.756392 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:45.756720 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:45.756725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:45.756728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:45.756944 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:45.757412 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:45.757419 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:45.757815 (+ 396us) client_negotiation.cc:769] Sending connection context
0618 22:33:45.758123 (+ 308us) client_negotiation.cc:240] Negotiation successful
0618 22:33:45.758636 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":243,"threads_started":1}
I20260618 22:33:45.761544 27502 heartbeater.cc:347] Connected to a master server at 127.18.150.190:42333
I20260618 22:33:45.761821 27502 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:45.762553 27502 heartbeater.cc:511] Master 127.18.150.190:42333 requested a full tablet report, sending...
I20260618 22:33:45.764521 27313 ts_manager.cc:194] Registered new tserver with Master: 4b3079f2f16e4d72a26539c993259cf6 (127.18.150.129:44067)
I20260618 22:33:45.765895 27313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:49047
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:33:45.797904 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:42333
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:46.122407 27508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:46.122735 27508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:46.122897 27508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:46.130366 27508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:46.130496 27508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:46.130652 27508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:33:46.140381 27508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:42333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:46.141678 27508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:46.143002 27508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:46.156517 27514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:46.156545 27516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:46.157191 27513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:46.157630 27508 server_base.cc:1034] running on GCE node
I20260618 22:33:46.158325 27508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:46.159215 27508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:46.160390 27508 hybrid_clock.cc:648] HybridClock initialized: now 1781822026160329 us; error 61 us; skew 500 ppm
I20260618 22:33:46.160645 27508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:46 dist-test-slave-h7jz krb5kdc[27268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822026, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:46.163702 27508 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:33:46.165133 27508 webserver.cc:460] Webserver started at http://127.18.150.130:35743/ using document root <none> and password file <none>
I20260618 22:33:46.165767 27508 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:46.165868 27508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:46.166167 27508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:46.168927 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "32e121dbcec94b1e9f37d6ab3f3b0903"
format_stamp: "Formatted at 2026-06-18 22:33:46 on dist-test-slave-h7jz"
server_key: "f0ea05961c74e5575645b42f0674e3e0"
server_key_iv: "89f0c894d23536977957dd2ce2cfb23c"
server_key_version: "encryptionkey@0"
I20260618 22:33:46.169660 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "32e121dbcec94b1e9f37d6ab3f3b0903"
format_stamp: "Formatted at 2026-06-18 22:33:46 on dist-test-slave-h7jz"
server_key: "f0ea05961c74e5575645b42f0674e3e0"
server_key_iv: "89f0c894d23536977957dd2ce2cfb23c"
server_key_version: "encryptionkey@0"
I20260618 22:33:46.174489 27508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:33:46.177747 27523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:46.179082 27508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:33:46.179222 27508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "32e121dbcec94b1e9f37d6ab3f3b0903"
format_stamp: "Formatted at 2026-06-18 22:33:46 on dist-test-slave-h7jz"
server_key: "f0ea05961c74e5575645b42f0674e3e0"
server_key_iv: "89f0c894d23536977957dd2ce2cfb23c"
server_key_version: "encryptionkey@0"
I20260618 22:33:46.179400 27508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:46.194070 27508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:46.198333 27508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:46.198589 27508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:46.199548 27508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:46.200912 27508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:46.200984 27508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:46.201093 27508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:46.201143 27508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:46.229645 27508 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:36243
I20260618 22:33:46.229674 27635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:36243 every 8 connection(s)
I20260618 22:33:46.230898 27508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:33:46.237079 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 27508
I20260618 22:33:46.237222 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:33:46.237619 19034 external_mini_cluster.cc:1428] Setting key dac02fbc365ecf7d7c6f9e052c5ec9ca
Jun 18 22:33:46 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822026, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:46.250056 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:46.233291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:38805 (local address 127.18.150.190:42333)
0618 22:33:46.233466 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:46.233470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:46.234715 (+ 1245us) server_negotiation.cc:407] Connection header received
0618 22:33:46.236491 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:46.236496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:46.236593 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:46.236679 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:46.238275 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.239132 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.240648 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.241059 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.241165 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:46.244571 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:46.244575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:46.244579 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:46.244609 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:46.247094 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:46.247924 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:46.247928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:46.247930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:46.248045 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:46.248459 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:46.248462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:46.248464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:46.248718 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:46.248828 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:46.249584 (+ 756us) server_negotiation.cc:299] Negotiation successful
0618 22:33:46.249774 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:33:46.251400 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:46.233776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:38805)
0618 22:33:46.234551 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:46.234590 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:46.236224 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:46.236935 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:46.236943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:46.237490 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:46.238118 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:46.238137 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.239293 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.239297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:46.240518 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:46.240529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.241284 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.241294 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:46.241512 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:46.242155 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:46.242176 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:46.244335 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:46.247234 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:46.247243 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:46.247262 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:46.247820 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:46.248128 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:46.248133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:46.248136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:46.248366 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:46.248831 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:46.248839 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:46.249240 (+ 401us) client_negotiation.cc:769] Sending connection context
0618 22:33:46.249519 (+ 279us) client_negotiation.cc:240] Negotiation successful
0618 22:33:46.250063 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":314,"threads_started":1}
I20260618 22:33:46.252866 27636 heartbeater.cc:347] Connected to a master server at 127.18.150.190:42333
I20260618 22:33:46.253175 27636 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:46.253887 27636 heartbeater.cc:511] Master 127.18.150.190:42333 requested a full tablet report, sending...
I20260618 22:33:46.255368 27313 ts_manager.cc:194] Registered new tserver with Master: 32e121dbcec94b1e9f37d6ab3f3b0903 (127.18.150.130:36243)
I20260618 22:33:46.256189 27313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:38805
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:33:46.284808 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:42333
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:46.590678 27641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:46.591004 27641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:46.591146 27641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:46.598479 27641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:46.598613 27641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:46.598764 27641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:33:46.608659 27641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:42333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:46.609987 27641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:46.611271 27641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:46.623884 27646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:46.623850 27647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:46.623922 27649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:46.624671 27641 server_base.cc:1034] running on GCE node
I20260618 22:33:46.625291 27641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:46.626262 27641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:46.627490 27641 hybrid_clock.cc:648] HybridClock initialized: now 1781822026627433 us; error 45 us; skew 500 ppm
I20260618 22:33:46.627815 27641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:46 dist-test-slave-h7jz krb5kdc[27268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822026, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:46.630841 27641 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:33:46.632270 27641 webserver.cc:460] Webserver started at http://127.18.150.131:38781/ using document root <none> and password file <none>
I20260618 22:33:46.632927 27641 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:46.633023 27641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:46.633339 27641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:33:46.636164 27641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "156389e2b3fa4df69a7dd321823b17d8"
format_stamp: "Formatted at 2026-06-18 22:33:46 on dist-test-slave-h7jz"
server_key: "5f5a9785f5c51839ee6650262037773f"
server_key_iv: "fd8d3cf6908a7b59447ba59daf967268"
server_key_version: "encryptionkey@0"
I20260618 22:33:46.636896 27641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "156389e2b3fa4df69a7dd321823b17d8"
format_stamp: "Formatted at 2026-06-18 22:33:46 on dist-test-slave-h7jz"
server_key: "5f5a9785f5c51839ee6650262037773f"
server_key_iv: "fd8d3cf6908a7b59447ba59daf967268"
server_key_version: "encryptionkey@0"
I20260618 22:33:46.642021 27641 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:33:46.645496 27656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:46.646941 27641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:33:46.647092 27641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "156389e2b3fa4df69a7dd321823b17d8"
format_stamp: "Formatted at 2026-06-18 22:33:46 on dist-test-slave-h7jz"
server_key: "5f5a9785f5c51839ee6650262037773f"
server_key_iv: "fd8d3cf6908a7b59447ba59daf967268"
server_key_version: "encryptionkey@0"
I20260618 22:33:46.647289 27641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:46.670439 27641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:46.674775 27641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:46.675033 27641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:46.676019 27641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:46.677364 27641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:46.677438 27641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:46.677541 27641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:46.677587 27641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:46.706215 27641 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:37215
I20260618 22:33:46.706246 27768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:37215 every 8 connection(s)
I20260618 22:33:46.707471 27641 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:33:46.713917 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 27641
I20260618 22:33:46.714090 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:33:46.714495 19034 external_mini_cluster.cc:1428] Setting key 7570bdafdfef3213c44c7a0c0a1d5d15
Jun 18 22:33:46 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822026, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:33:46.726186 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:46.709840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:35075 (local address 127.18.150.190:42333)
0618 22:33:46.710033 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:46.710038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:46.711168 (+ 1130us) server_negotiation.cc:407] Connection header received
0618 22:33:46.712958 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:46.712963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:46.713067 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:46.713152 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:46.714707 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.715534 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.717146 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.717634 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.717753 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:46.720915 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:46.720920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:46.720923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:46.720953 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:46.723427 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:46.724130 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:46.724134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:46.724142 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:46.724291 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:46.724660 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:46.724664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:46.724667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:46.724923 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:46.725030 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:46.725716 (+ 686us) server_negotiation.cc:299] Negotiation successful
0618 22:33:46.725880 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:33:46.727130 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:46.710365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:35075)
0618 22:33:46.711021 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:46.711058 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:46.712697 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:46.713393 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:46.713401 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:46.713904 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:46.714541 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:46.714564 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.715671 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.715675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:46.717024 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:46.717039 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.717829 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.717834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:46.718089 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:46.718657 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:46.718678 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:46.720731 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:33:46.723564 (+ 2833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:46.723571 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:46.723586 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:46.724023 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:46.724392 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:33:46.724395 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:33:46.724397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:33:46.724578 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:33:46.725013 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:33:46.725019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:33:46.725353 (+ 334us) client_negotiation.cc:769] Sending connection context
0618 22:33:46.725591 (+ 238us) client_negotiation.cc:240] Negotiation successful
0618 22:33:46.726013 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":241,"threads_started":1}
I20260618 22:33:46.728505 27769 heartbeater.cc:347] Connected to a master server at 127.18.150.190:42333
I20260618 22:33:46.728804 27769 heartbeater.cc:464] Registering TS with master...
I20260618 22:33:46.729557 27769 heartbeater.cc:511] Master 127.18.150.190:42333 requested a full tablet report, sending...
I20260618 22:33:46.730938 27312 ts_manager.cc:194] Registered new tserver with Master: 156389e2b3fa4df69a7dd321823b17d8 (127.18.150.131:37215)
I20260618 22:33:46.731741 27312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:35075
I20260618 22:33:46.741596 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.129@KRBTEST.COM" created.
I20260618 22:33:46.768782 27502 heartbeater.cc:503] Master 127.18.150.190:42333 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab.
Entry for principal oryx/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab.
WARNING: no policy specified for oryx/127.18.150.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab.
Entry for principal oryx/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab.
WARNING: no policy specified for oryx/127.18.150.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab.
Entry for principal oryx/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab.
I20260618 22:33:46.864823 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 27374
Jun 18 22:33:46 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:33:46.880846 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:46.865896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59174 (local address 127.18.150.129:44067)
0618 22:33:46.866502 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:46.866509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:46.866537 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:33:46.866612 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:46.866619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:46.866840 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:46.866975 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:46.868445 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.869535 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.870845 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.871406 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.871500 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:46.875200 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:46.875211 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:46.875226 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:46.875262 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:46.877896 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:46.878588 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:46.878598 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:46.878601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:46.878745 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:46.879112 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:46.879116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:46.879118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:46.879537 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:46.879826 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:46.880183 (+ 357us) server_negotiation.cc:299] Negotiation successful
0618 22:33:46.880453 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":316,"threads_started":1}
I20260618 22:33:46.882630 27497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:46.963054 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 27374
I20260618 22:33:46.975802 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 27508
Jun 18 22:33:46 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:33:46.991215 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:46.976866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41754 (local address 127.18.150.130:36243)
0618 22:33:46.977399 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:46.977406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:46.977432 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:33:46.977489 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:46.977495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:46.977715 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:46.977843 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:46.979049 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.980008 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.981211 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:46.981773 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:46.981870 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:46.985882 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:46.985892 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:46.985906 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:46.985967 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:46.988412 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:46.989018 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:46.989025 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:46.989027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:46.989139 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:46.989530 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:46.989535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:46.989538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:46.989876 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:46.990244 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:46.990622 (+ 378us) server_negotiation.cc:299] Negotiation successful
0618 22:33:46.990886 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":294,"threads_started":1}
I20260618 22:33:46.992369 27631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:47.077075 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 27508
I20260618 22:33:47.090250 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 27641
Jun 18 22:33:47 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:33:47.108285 27783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:47.091229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50022 (local address 127.18.150.131:37215)
0618 22:33:47.091839 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:47.091846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:47.091875 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:33:47.091934 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:47.091940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:47.092161 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:47.092315 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:47.093708 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:47.094867 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:47.096349 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:47.097097 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:47.097224 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:47.101916 (+ 4692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:47.101933 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:47.101983 (+ 50us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:47.102044 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:47.105102 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:47.105989 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:47.106001 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:47.106004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:47.106182 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:47.106601 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:47.106606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:47.106610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:47.107004 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:47.107306 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:47.107643 (+ 337us) server_negotiation.cc:299] Negotiation successful
0618 22:33:47.107941 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":305,"threads_started":1}
I20260618 22:33:47.109653 27764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:47.190729 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 27641
I20260618 22:33:47.202994 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 27281
I20260618 22:33:47.204058 27343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:33:47.305382 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 27281
I20260618 22:33:47.321712 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:42333
--webserver_interface=127.18.150.190
--webserver_port=35689
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:42333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:33:47.632999 27786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:47.633294 27786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:47.633428 27786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:47.640805 27786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:33:47.640965 27786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:47.641055 27786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:47.641130 27786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:33:47.641191 27786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:33:47.651326 27786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:42333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:42333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=35689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:47.652671 27786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:47.654047 27786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:33:47.665987 27786 server_base.cc:1034] running on GCE node
W20260618 22:33:47.665933 27792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:47.666024 27791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:47.666169 27794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:47.667174 27786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:47.668589 27786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:47.669822 27786 hybrid_clock.cc:648] HybridClock initialized: now 1781822027669761 us; error 46 us; skew 500 ppm
I20260618 22:33:47.670193 27786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:47 dist-test-slave-h7jz krb5kdc[27268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822027, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:47.673123 27786 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:33:47.674618 27786 webserver.cc:460] Webserver started at http://127.18.150.190:35689/ using document root <none> and password file <none>
I20260618 22:33:47.675288 27786 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:47.675413 27786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:47.680718 27786 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:33:47.683524 27801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:47.684930 27786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:33:47.685097 27786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "74d1f5798cc942f99ef91d06eda38dff"
format_stamp: "Formatted at 2026-06-18 22:33:45 on dist-test-slave-h7jz"
server_key: "8d5864ff6a70419875fd8c322caca4ab"
server_key_iv: "c5265d576c85b67ba17ea80fcad065ab"
server_key_version: "encryptionkey@0"
I20260618 22:33:47.686077 27786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:47.705834 27786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:47.709969 27786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:47.710227 27786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:47.729962 27786 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:42333
I20260618 22:33:47.729986 27852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:42333 every 8 connection(s)
I20260618 22:33:47.731891 27786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:33:47.739078 27853 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:33:47.741616 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 27786
I20260618 22:33:47.742578 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:44067
--local_ip_for_outbound_sockets=127.18.150.129
--tserver_master_addrs=127.18.150.190:42333
--webserver_port=35551
--webserver_interface=127.18.150.129
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.150.129
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:33:47.742637 27853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff: Bootstrap starting.
I20260618 22:33:47.762732 27853 log.cc:824] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff: Log is configured to *not* fsync() on all Append() calls
I20260618 22:33:47.771405 27853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:33:47.771884 27853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff: Bootstrap complete.
I20260618 22:33:47.780527 27853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } }
I20260618 22:33:47.781304 27853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74d1f5798cc942f99ef91d06eda38dff, State: Initialized, Role: FOLLOWER
I20260618 22:33:47.781899 27853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } }
I20260618 22:33:47.782140 27853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:33:47.782245 27853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:33:47.782407 27853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:33:47.786105 27853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } }
I20260618 22:33:47.786590 27853 leader_election.cc:302] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74d1f5798cc942f99ef91d06eda38dff; no voters:
I20260618 22:33:47.787132 27853 leader_election.cc:288] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:33:47.787267 27857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:33:47.788406 27857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [term 2 LEADER]: Becoming Leader. State: Replica: 74d1f5798cc942f99ef91d06eda38dff, State: Running, Role: LEADER
I20260618 22:33:47.788973 27857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } }
I20260618 22:33:47.789242 27853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:33:47.791611 27858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "74d1f5798cc942f99ef91d06eda38dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } } }
I20260618 22:33:47.792021 27858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:47.791584 27859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74d1f5798cc942f99ef91d06eda38dff. Latest consensus state: current_term: 2 leader_uuid: "74d1f5798cc942f99ef91d06eda38dff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74d1f5798cc942f99ef91d06eda38dff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42333 } } }
I20260618 22:33:47.792817 27859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff [sys.catalog]: This master's current role is: LEADER
I20260618 22:33:47.794966 27864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:33:47.800561 27864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:33:47.801913 27864 catalog_manager.cc:1259] Loaded cluster ID: dfc94adad3754baca973c1b327d326ff
I20260618 22:33:47.802107 27864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:33:47.806046 27864 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:33:47.807498 27864 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 74d1f5798cc942f99ef91d06eda38dff: Loaded TSK: 0
I20260618 22:33:47.809051 27864 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:33:48.068384 27855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:48.068723 27855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:48.068858 27855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:48.076164 27855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:48.076290 27855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:48.076434 27855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:33:48.086396 27855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.129.keytab
--principal=oryx/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:44067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=35551
--tserver_master_addrs=127.18.150.190:42333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:48.087702 27855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:48.088963 27855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:48.101615 27880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:48.101634 27879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:48.102221 27882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:48.102766 27855 server_base.cc:1034] running on GCE node
I20260618 22:33:48.103367 27855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:48.104378 27855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:48.105590 27855 hybrid_clock.cc:648] HybridClock initialized: now 1781822028105539 us; error 39 us; skew 500 ppm
I20260618 22:33:48.105978 27855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822028, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:48.109122 27855 init.cc:377] Logged in from keytab as oryx/127.18.150.129@KRBTEST.COM (short username oryx)
I20260618 22:33:48.110671 27855 webserver.cc:460] Webserver started at http://127.18.150.129:35551/ using document root <none> and password file <none>
I20260618 22:33:48.111344 27855 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:48.111465 27855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:48.116838 27855 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:33:48.119531 27889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:48.120854 27855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260618 22:33:48.120996 27855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "4b3079f2f16e4d72a26539c993259cf6"
format_stamp: "Formatted at 2026-06-18 22:33:45 on dist-test-slave-h7jz"
server_key: "13c22d4c1c4b623a99290b3c1f3f1c55"
server_key_iv: "d1b356af4eac04e4d6c7f0dcfa6f7f31"
server_key_version: "encryptionkey@0"
I20260618 22:33:48.121883 27855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:48.137171 27855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:48.141170 27855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:48.141423 27855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:48.142413 27855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:48.143746 27855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:48.143834 27855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:48.143932 27855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:48.144001 27855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:48.173290 27855 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:44067
I20260618 22:33:48.173355 28001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:44067 every 8 connection(s)
I20260618 22:33:48.174726 27855 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:33:48.175347 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 27855
I20260618 22:33:48.176081 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:36243
--local_ip_for_outbound_sockets=127.18.150.130
--tserver_master_addrs=127.18.150.190:42333
--webserver_port=35743
--webserver_interface=127.18.150.130
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.150.130
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:33:48.196894 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:48.177764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:46223 (local address 127.18.150.190:42333)
0618 22:33:48.182470 (+ 4706us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:48.182490 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:48.182530 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:33:48.182879 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:48.182900 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:48.183302 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:48.183622 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:48.185500 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.186808 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.188904 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.189740 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.189887 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.195957 (+ 6070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:46223: BlockingRecv error: recv got EOF from 127.18.150.129:46223 (error 108)
Metrics: {"server-negotiator.queue_time_us":4480,"thread_start_us":4262,"threads_started":1}
I20260618 22:33:48.196931 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:48.177772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:46223)
0618 22:33:48.178607 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:48.178658 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:48.180738 (+ 2080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:48.183824 (+ 3086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:48.183834 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:48.184544 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:48.185271 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.185295 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.186970 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.186975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.188742 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.188758 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.189980 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.189987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.190341 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.191143 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:48.191170 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:48.195649 (+ 4479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":330,"threads_started":1}
W20260618 22:33:48.197563 28002 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:42333 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.150.190:42333: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:48.205272 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:48.198178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:38647)
0618 22:33:48.198356 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:48.198375 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:48.198489 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:48.198808 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:48.198812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:48.199142 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:48.199277 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.199285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.200235 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.200240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.201110 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.201131 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.201747 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.201752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.201917 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.202455 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:48.202472 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:48.205007 (+ 2535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260618 22:33:48.205392 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:48.198248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:38647 (local address 127.18.150.190:42333)
0618 22:33:48.198386 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:48.198391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:48.198427 (+ 36us) server_negotiation.cc:407] Connection header received
0618 22:33:48.198596 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:48.198601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:48.198688 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:48.198766 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:48.199404 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.200106 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.201244 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.201622 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.201723 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.205196 (+ 3473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:38647: BlockingRecv error: recv got EOF from 127.18.150.129:38647 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:48.213047 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:48.205970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:48885)
0618 22:33:48.206106 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:48.206124 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:48.206213 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:48.206521 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:48.206525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:48.206804 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:48.206934 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.206942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.207854 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.207859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.208734 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.208745 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.209361 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.209366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.209542 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.210096 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:48.210112 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:48.212779 (+ 2667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260618 22:33:48.213249 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:48.206126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:48885 (local address 127.18.150.190:42333)
0618 22:33:48.206281 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:48.206288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:48.206303 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:33:48.206345 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:48.206350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:48.206411 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:48.206486 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:48.207058 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.207727 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.208856 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.209242 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.209339 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.212968 (+ 3629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:48885: BlockingRecv error: recv got EOF from 127.18.150.129:48885 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260618 22:33:48.213341 28002 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:33:48.486373 28006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:48.486649 28006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:48.486780 28006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:48.494010 28006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:48.494124 28006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:48.494308 28006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:33:48.503643 28006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.130.keytab
--principal=oryx/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:36243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=35743
--tserver_master_addrs=127.18.150.190:42333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:48.504958 28006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:48.506237 28006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:48.518261 28011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:48.518365 28012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:48.518288 28014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:48.519709 28006 server_base.cc:1034] running on GCE node
I20260618 22:33:48.520324 28006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:48.521281 28006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:48.522478 28006 hybrid_clock.cc:648] HybridClock initialized: now 1781822028522421 us; error 46 us; skew 500 ppm
I20260618 22:33:48.522780 28006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822028, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:48.525754 28006 init.cc:377] Logged in from keytab as oryx/127.18.150.130@KRBTEST.COM (short username oryx)
I20260618 22:33:48.527225 28006 webserver.cc:460] Webserver started at http://127.18.150.130:35743/ using document root <none> and password file <none>
I20260618 22:33:48.527878 28006 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:48.527981 28006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:48.533100 28006 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260618 22:33:48.535686 28021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:48.537099 28006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:33:48.537245 28006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "32e121dbcec94b1e9f37d6ab3f3b0903"
format_stamp: "Formatted at 2026-06-18 22:33:46 on dist-test-slave-h7jz"
server_key: "f0ea05961c74e5575645b42f0674e3e0"
server_key_iv: "89f0c894d23536977957dd2ce2cfb23c"
server_key_version: "encryptionkey@0"
I20260618 22:33:48.538205 28006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:48.557703 28006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:48.561793 28006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:48.562063 28006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:48.562986 28006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:48.564218 28006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:48.564299 28006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:48.564412 28006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:48.564476 28006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:48.594091 28006 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:36243
I20260618 22:33:48.594128 28133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:36243 every 8 connection(s)
I20260618 22:33:48.595463 28006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:33:48.598389 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 28006
I20260618 22:33:48.599102 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:37215
--local_ip_for_outbound_sockets=127.18.150.131
--tserver_master_addrs=127.18.150.190:42333
--webserver_port=38781
--webserver_interface=127.18.150.131
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.150.131
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:33:48.611055 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:48.597907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:41175 (local address 127.18.150.190:42333)
0618 22:33:48.598092 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:48.598096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:48.599577 (+ 1481us) server_negotiation.cc:407] Connection header received
0618 22:33:48.601375 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:48.601381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:48.601459 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:48.601543 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:48.603142 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.603948 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.605597 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.606025 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.606135 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.610821 (+ 4686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:41175: BlockingRecv error: recv got EOF from 127.18.150.130:41175 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260618 22:33:48.611783 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:48.598612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:41175)
0618 22:33:48.599419 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:48.599467 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:48.601105 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:48.601794 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:48.601803 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:48.602363 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:48.602947 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.602966 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.604102 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.604106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.605463 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.605475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.606238 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.606245 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.606468 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.607094 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:48.607119 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:48.610613 (+ 3494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":289,"threads_started":1}
W20260618 22:33:48.612525 28134 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:42333 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.150.190:42333: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:48.620185 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:48.613112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:50001)
0618 22:33:48.613285 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:48.613300 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:48.613399 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:48.613677 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:48.613681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:48.614018 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:48.614152 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.614160 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.615099 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.615103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.615967 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.615977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.616601 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.616606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.616757 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.617245 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:48.617261 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:48.619932 (+ 2671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260618 22:33:48.620647 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:48.613273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:50001 (local address 127.18.150.190:42333)
0618 22:33:48.613430 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:48.613436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:48.613452 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:33:48.613507 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:48.613510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:48.613563 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:48.613639 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:48.614275 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.614977 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.616085 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.616476 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.616575 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.620135 (+ 3560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:50001: BlockingRecv error: recv got EOF from 127.18.150.130:50001 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:48.627889 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:48.620833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:59911)
0618 22:33:48.620969 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:48.620985 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:48.621066 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:48.621550 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:48.621553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:48.621798 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:48.621932 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.621968 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.622862 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.622866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.623734 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:48.623744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.624341 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.624346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:48.624529 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.625036 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:48.625052 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:48.627549 (+ 2497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260618 22:33:48.627954 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:48.621136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:59911 (local address 127.18.150.190:42333)
0618 22:33:48.621287 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:48.621293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:48.621309 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:33:48.621357 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:48.621362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:48.621427 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:48.621512 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:48.622078 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.622734 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.623851 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:48.624219 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:48.624315 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:48.627737 (+ 3422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:59911: BlockingRecv error: recv got EOF from 127.18.150.130:59911 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260618 22:33:48.628281 28134 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:33:48.912241 28137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:33:48.912547 28137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:33:48.912704 28137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:33:48.920395 28137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:33:48.920524 28137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:33:48.920707 28137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:33:48.930665 28137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/krb5kdc/oryx_127.18.150.131.keytab
--principal=oryx/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:37215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=38781
--tserver_master_addrs=127.18.150.190:42333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:33:48.932034 28137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:33:48.933353 28137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:33:48.946349 28142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:48.946427 28145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:33:48.946552 28143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:33:48.947048 28137 server_base.cc:1034] running on GCE node
I20260618 22:33:48.947674 28137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:33:48.948606 28137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:33:48.949791 28137 hybrid_clock.cc:648] HybridClock initialized: now 1781822028949746 us; error 40 us; skew 500 ppm
I20260618 22:33:48.950132 28137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:33:48 dist-test-slave-h7jz krb5kdc[27268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822028, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:33:48.953115 28137 init.cc:377] Logged in from keytab as oryx/127.18.150.131@KRBTEST.COM (short username oryx)
I20260618 22:33:48.954576 28137 webserver.cc:460] Webserver started at http://127.18.150.131:38781/ using document root <none> and password file <none>
I20260618 22:33:48.955281 28137 fs_manager.cc:359] Metadata directory not provided
I20260618 22:33:48.955411 28137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:33:48.960723 28137 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:33:48.963419 28152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:48.964748 28137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260618 22:33:48.964890 28137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "156389e2b3fa4df69a7dd321823b17d8"
format_stamp: "Formatted at 2026-06-18 22:33:46 on dist-test-slave-h7jz"
server_key: "5f5a9785f5c51839ee6650262037773f"
server_key_iv: "fd8d3cf6908a7b59447ba59daf967268"
server_key_version: "encryptionkey@0"
I20260618 22:33:48.965816 28137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:33:48.982707 28137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:33:48.986647 28137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:33:48.986896 28137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:33:48.987835 28137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:33:48.989184 28137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:33:48.989280 28137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:48.989380 28137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:33:48.989456 28137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:33:49.017915 28137 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:37215
I20260618 22:33:49.017978 28264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:37215 every 8 connection(s)
I20260618 22:33:49.019304 28137 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:33:49.029099 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 28137
Jun 18 22:33:49 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:49 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:33:49.034885 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:49.021666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:51327 (local address 127.18.150.190:42333)
0618 22:33:49.021862 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:49.021868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:49.023048 (+ 1180us) server_negotiation.cc:407] Connection header received
0618 22:33:49.024772 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:49.024776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:49.024845 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:49.024925 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:49.026460 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.027148 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.028683 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.029067 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.029165 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:49.034646 (+ 5481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:51327: BlockingRecv error: recv got EOF from 127.18.150.131:51327 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:33:49.035607 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:49.022247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:51327)
0618 22:33:49.022898 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:49.022936 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:49.024546 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:49.025148 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:49.025157 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:49.025697 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:49.026308 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:49.026326 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.027315 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.027320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:49.028559 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:49.028571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.029305 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.029313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:49.029608 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:49.030327 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:49.030359 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:49.034471 (+ 4112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":240,"threads_started":1}
W20260618 22:33:49.036248 28265 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:42333 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.150.190:42333: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Jun 18 22:33:49 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:49 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:49.044530 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:49.036775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:49997)
0618 22:33:49.036942 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:49.036960 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:49.037044 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:49.037326 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:49.037329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:49.037589 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:49.037708 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:49.037715 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.038970 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.038974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:49.039818 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:49.039828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.040568 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.040573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:49.040726 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:49.041157 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:49.041172 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:49.044256 (+ 3084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260618 22:33:49.044699 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:49.036917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:49997 (local address 127.18.150.190:42333)
0618 22:33:49.037063 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:49.037068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:49.037085 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:33:49.037139 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:49.037144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:49.037212 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:49.037309 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:49.037834 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.038864 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.039966 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.040434 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.040542 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:49.044474 (+ 3932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:49997: BlockingRecv error: recv got EOF from 127.18.150.131:49997 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:33:49.045922 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:49.030163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.18.150.190:42333)
0618 22:33:49.030829 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:49.030835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:49.030852 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:33:49.030895 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:49.030900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:49.030966 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:49.031068 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:49.032931 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.034079 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.035608 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.036125 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.036230 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:49.039199 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:33:49.039207 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:33:49.039224 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:33:49.039259 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:33:49.042420 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:49.043258 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:49.043268 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:49.043278 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:49.043515 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:33:49.043910 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:33:49.043913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:33:49.043916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:33:49.044404 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:33:49.044705 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0618 22:33:49.045199 (+ 494us) server_negotiation.cc:299] Negotiation successful
0618 22:33:49.045545 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":449,"threads_started":1}
Jun 18 22:33:49 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:49 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:49.053090 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:49.045357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:38793)
0618 22:33:49.045582 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:49.045599 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:49.045690 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:49.046017 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:49.046022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:49.046360 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:49.046500 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:49.046507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.047649 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.047655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:49.048742 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:49.048756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.049498 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.049504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:49.049671 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:49.050266 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:49.050285 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:49.052818 (+ 2533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129}
W20260618 22:33:49.053236 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:49.045410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:38793 (local address 127.18.150.190:42333)
0618 22:33:49.045643 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:49.045661 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:49.045679 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:33:49.045794 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:49.045798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:49.045865 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:49.045996 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:49.046619 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.047519 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.048871 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.049364 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.049470 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:49.053014 (+ 3544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:38793: BlockingRecv error: recv got EOF from 127.18.150.131:38793 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
W20260618 22:33:49.053431 28265 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 18 22:33:49 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:49 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:49.221231 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:49.213882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:48339)
0618 22:33:49.214399 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:49.214420 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:49.214532 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:49.214839 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:49.214842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:49.215252 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:49.215430 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:49.215438 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.216335 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.216338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:49.217275 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:49.217286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.217912 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.217917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:49.218066 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:49.218562 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:49.218577 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:49.221015 (+ 2438us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":273,"threads_started":1}
W20260618 22:33:49.221400 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:49.213918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:48339 (local address 127.18.150.190:42333)
0618 22:33:49.214094 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:49.214099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:49.214486 (+ 387us) server_negotiation.cc:407] Connection header received
0618 22:33:49.214634 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:49.214638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:49.214712 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:49.214805 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:49.215535 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.216216 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.217377 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.217768 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.217851 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:49.221204 (+ 3353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:48339: BlockingRecv error: recv got EOF from 127.18.150.129:48339 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 18 22:33:49 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:49 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:49.636615 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:49.628841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:56789)
0618 22:33:49.629400 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:49.629416 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:49.629546 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:49.629867 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:49.629872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:49.630331 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:49.630507 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:49.630516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.631563 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.631569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:49.632512 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:49.632523 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.633179 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.633185 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:49.633334 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:49.633845 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:49.633861 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:49.636393 (+ 2532us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":331,"threads_started":1}
W20260618 22:33:49.636757 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:49.628971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:56789 (local address 127.18.150.190:42333)
0618 22:33:49.629238 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:49.629244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:49.629501 (+ 257us) server_negotiation.cc:407] Connection header received
0618 22:33:49.629660 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:49.629665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:49.629730 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:49.629811 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:49.630646 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.631434 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.632640 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:49.633025 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:49.633124 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:49.636565 (+ 3441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:56789: BlockingRecv error: recv got EOF from 127.18.150.130:56789 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Jun 18 22:33:50 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:50 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:50.062709 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:50.054044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:34971)
0618 22:33:50.054542 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:50.054562 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:50.054690 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:50.055018 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:50.055023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:50.055474 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:50.055676 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:50.055689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.056750 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.056762 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:50.057731 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:50.057772 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.058616 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.058622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:50.058789 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:50.059506 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:50.059530 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:50.062482 (+ 2952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":277,"threads_started":1}
W20260618 22:33:50.062856 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:50.054148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:34971 (local address 127.18.150.190:42333)
0618 22:33:50.054329 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:50.054335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:50.054635 (+ 300us) server_negotiation.cc:407] Connection header received
0618 22:33:50.054827 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:50.054831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:50.054892 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:50.054969 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:50.055815 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.056530 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.057911 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.058424 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.058532 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:50.062657 (+ 4125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:34971: BlockingRecv error: recv got EOF from 127.18.150.131:34971 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 18 22:33:50 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:50 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:50.230269 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:50.222102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:58559)
0618 22:33:50.222560 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:50.222580 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:50.222706 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:50.223031 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:50.223035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:50.223414 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:50.223576 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:50.223585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.224648 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.224653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:50.225970 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:50.225986 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.226626 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.226632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:50.226796 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:50.227347 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:50.227363 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:50.229918 (+ 2555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":241,"threads_started":1}
W20260618 22:33:50.230365 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:50.222139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:58559 (local address 127.18.150.190:42333)
0618 22:33:50.222383 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:50.222389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:50.222662 (+ 273us) server_negotiation.cc:407] Connection header received
0618 22:33:50.222844 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:50.222849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:50.222914 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:50.222993 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:50.223711 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.224532 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.226098 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.226494 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.226597 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:50.230134 (+ 3537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:58559: BlockingRecv error: recv got EOF from 127.18.150.129:58559 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Jun 18 22:33:50 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:50 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:50.645272 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:50.637543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:47217)
0618 22:33:50.638084 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:50.638100 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:50.638228 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:50.638553 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:50.638558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:50.638948 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:50.639114 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:50.639123 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.640122 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.640127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:50.641088 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:50.641105 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.641860 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.641865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:50.642005 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:50.642538 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:50.642555 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:50.645033 (+ 2478us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":291,"threads_started":1}
W20260618 22:33:50.645406 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:50.637681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:47217 (local address 127.18.150.190:42333)
0618 22:33:50.637888 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:50.637895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:50.638178 (+ 283us) server_negotiation.cc:407] Connection header received
0618 22:33:50.638359 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:50.638363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:50.638422 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:50.638507 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:50.639246 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.639982 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.641229 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:50.641731 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:50.641824 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:50.645209 (+ 3385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:47217: BlockingRecv error: recv got EOF from 127.18.150.130:47217 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 18 22:33:51 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:51 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:51.074149 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:51.063562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:48115)
0618 22:33:51.064110 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:51.064127 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:51.064237 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:51.064508 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:51.064511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:51.064900 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:51.065059 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:51.065067 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.066019 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.066023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:51.066904 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:51.066915 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.067542 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.067546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:51.067652 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:51.070989 (+ 3337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:51.071007 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:51.073862 (+ 2855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":320,"threads_started":1}
W20260618 22:33:51.074261 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:51.063705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:48115 (local address 127.18.150.190:42333)
0618 22:33:51.063879 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:51.063883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:51.064190 (+ 307us) server_negotiation.cc:407] Connection header received
0618 22:33:51.064331 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:51.064334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:51.064405 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:51.064481 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:51.065168 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.065868 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.067023 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.067431 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.067520 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:51.074060 (+ 6540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:48115: BlockingRecv error: recv got EOF from 127.18.150.131:48115 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 18 22:33:51 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:51 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:51.239217 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:51.231214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:60567)
0618 22:33:51.231690 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:51.231711 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:51.231867 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:51.232181 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:51.232185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:51.232524 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:51.232682 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:51.232691 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.233712 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.233717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:51.234821 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:51.234835 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.235577 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.235583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:51.235762 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:51.236325 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:51.236343 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:51.238971 (+ 2628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":268,"threads_started":1}
W20260618 22:33:51.239355 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:51.231290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:60567 (local address 127.18.150.190:42333)
0618 22:33:51.231466 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:51.231472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:51.231819 (+ 347us) server_negotiation.cc:407] Connection header received
0618 22:33:51.231987 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:51.231992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:51.232058 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:51.232148 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:51.232830 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.233569 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.234932 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.235409 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.235526 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:51.239142 (+ 3616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:60567: BlockingRecv error: recv got EOF from 127.18.150.129:60567 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 18 22:33:51 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:51 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:51.654098 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:51.646127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:57059)
0618 22:33:51.646544 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:51.646560 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:51.646689 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:51.646989 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:51.646994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:51.647335 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:51.647495 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:51.647504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.648499 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.648503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:51.649380 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:51.649391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.650052 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.650057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:51.650232 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:51.650816 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:51.650834 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:51.653698 (+ 2864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":223,"threads_started":1}
W20260618 22:33:51.654155 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:51.646176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:57059 (local address 127.18.150.190:42333)
0618 22:33:51.646354 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:51.646360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:51.646662 (+ 302us) server_negotiation.cc:407] Connection header received
0618 22:33:51.646797 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:51.646802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:51.646869 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:51.646945 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:51.647640 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.648371 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.649502 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:51.649910 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:51.650078 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:51.653915 (+ 3837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:57059: BlockingRecv error: recv got EOF from 127.18.150.130:57059 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 18 22:33:52 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:52 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:52.082223 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:52.074962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:49035)
0618 22:33:52.075438 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:52.075458 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:52.075567 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:52.075855 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:52.075858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:52.076166 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:52.076328 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:52.076336 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.077299 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.077303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:52.078199 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:52.078211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.078800 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.078804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:52.078922 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:52.079448 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:52.079464 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:52.081968 (+ 2504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":265,"threads_started":1}
W20260618 22:33:52.082362 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:52.075055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:49035 (local address 127.18.150.190:42333)
0618 22:33:52.075205 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:52.075210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:52.075520 (+ 310us) server_negotiation.cc:407] Connection header received
0618 22:33:52.075649 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:52.075653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:52.075722 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:52.075821 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:52.076456 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.077177 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.078310 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.078699 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.078793 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:52.082174 (+ 3381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:49035: BlockingRecv error: recv got EOF from 127.18.150.131:49035 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 18 22:33:52 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:52 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:52.248145 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:52.240157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:53119)
0618 22:33:52.240653 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:52.240677 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:52.240791 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:52.241118 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:52.241123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:52.241496 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:52.241670 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:52.241683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.242683 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.242689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:52.243669 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:52.243681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.244368 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.244374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:52.244540 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:52.245111 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:52.245129 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:52.247916 (+ 2787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":237,"threads_started":1}
W20260618 22:33:52.248451 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:52.240266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:53119 (local address 127.18.150.190:42333)
0618 22:33:52.240488 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:52.240493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:52.240765 (+ 272us) server_negotiation.cc:407] Connection header received
0618 22:33:52.240904 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:52.240908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:52.240967 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:52.241060 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:52.241813 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.242563 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.243809 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.244217 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.244336 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:52.248182 (+ 3846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:53119: BlockingRecv error: recv got EOF from 127.18.150.129:53119 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Jun 18 22:33:52 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:52 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:52.662851 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:52.655260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:49403)
0618 22:33:52.655719 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:52.655737 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:52.655858 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:52.656148 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:52.656153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:52.656559 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:52.656699 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:52.656708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.657620 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.657625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:52.658602 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:52.658613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.659254 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.659259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:52.659435 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:52.659982 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:52.659998 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:52.662632 (+ 2634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":229,"threads_started":1}
W20260618 22:33:52.662986 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:52.655302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:49403 (local address 127.18.150.190:42333)
0618 22:33:52.655480 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:52.655486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:52.655834 (+ 348us) server_negotiation.cc:407] Connection header received
0618 22:33:52.655963 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:52.655968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:52.656027 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:52.656098 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:52.656839 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.657500 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.658729 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:52.659130 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:52.659229 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:52.662794 (+ 3565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:49403: BlockingRecv error: recv got EOF from 127.18.150.130:49403 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 18 22:33:53 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:53 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:53.091068 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:53.083089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:35333)
0618 22:33:53.083537 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:53.083553 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:53.083674 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:53.083974 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:53.083977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:53.084444 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:53.084596 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:53.084605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.085847 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.085852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:53.086889 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:53.086901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.087510 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.087515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:53.087631 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:53.088184 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:53.088199 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:53.090851 (+ 2652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":239,"threads_started":1}
W20260618 22:33:53.091202 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:53.083177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:35333 (local address 127.18.150.190:42333)
0618 22:33:53.083360 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:53.083365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:53.083623 (+ 258us) server_negotiation.cc:407] Connection header received
0618 22:33:53.083765 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:53.083768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:53.083852 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:53.083944 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:53.084721 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.085679 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.087012 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.087392 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.087480 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:53.091006 (+ 3526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:35333: BlockingRecv error: recv got EOF from 127.18.150.131:35333 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Jun 18 22:33:53 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:53 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:53.257299 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:53.249181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:38345)
0618 22:33:53.249647 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:53.249668 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:53.249788 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:53.250105 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:53.250108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:53.250515 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:53.250669 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:53.250677 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.251880 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.251887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:53.252843 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:53.252856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.253552 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.253558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:53.253737 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:53.254351 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:53.254370 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:53.257032 (+ 2662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":225,"threads_started":1}
W20260618 22:33:53.257391 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:53.249296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:38345 (local address 127.18.150.190:42333)
0618 22:33:53.249518 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:53.249523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:53.249750 (+ 227us) server_negotiation.cc:407] Connection header received
0618 22:33:53.249889 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:53.249893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:53.249977 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:53.250054 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:53.250849 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.251716 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.252991 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.253404 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.253523 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:53.257211 (+ 3688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:38345: BlockingRecv error: recv got EOF from 127.18.150.129:38345 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 18 22:33:53 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:53 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:53.671684 28283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:53.663693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:49657)
0618 22:33:53.664216 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:53.664233 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:53.664378 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:53.664695 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:53.664700 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:53.665128 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:53.665293 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:53.665302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.666334 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.666340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:53.667305 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:53.667316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.667970 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.667975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:53.668123 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:53.668657 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:53.668676 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:53.671370 (+ 2694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":228,"threads_started":1}
W20260618 22:33:53.671782 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:53.663936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:49657 (local address 127.18.150.190:42333)
0618 22:33:53.664154 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:53.664161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:53.664310 (+ 149us) server_negotiation.cc:407] Connection header received
0618 22:33:53.664502 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:53.664507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:53.664567 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:53.664647 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:53.665430 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.666211 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.667422 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:53.667834 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:53.667935 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:53.671546 (+ 3611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:49657: BlockingRecv error: recv got EOF from 127.18.150.130:49657 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Jun 18 22:33:54 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:54 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:54.100104 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:54.091911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:43769)
0618 22:33:54.092373 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:54.092389 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:54.092505 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:54.092790 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:54.092795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:54.093280 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:54.093439 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:54.093448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.094583 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.094588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:54.095481 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:54.095492 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.096127 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.096132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:54.096288 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:54.096846 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:54.096861 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:54.099764 (+ 2903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":224,"threads_started":1}
W20260618 22:33:54.100214 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:54.091987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:43769 (local address 127.18.150.190:42333)
0618 22:33:54.092164 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:54.092170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:54.092461 (+ 291us) server_negotiation.cc:407] Connection header received
0618 22:33:54.092606 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:54.092611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:54.092677 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:54.092754 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:54.093576 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.094458 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.095607 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.096002 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.096102 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:54.099967 (+ 3865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:43769: BlockingRecv error: recv got EOF from 127.18.150.131:43769 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 18 22:33:54 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:54 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:54.265743 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:54.258187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:52747)
0618 22:33:54.258679 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:54.258701 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:54.258812 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:54.259071 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:54.259074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:54.259438 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:54.259593 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:54.259601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.260558 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.260562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:54.261460 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:54.261472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.262172 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.262178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:54.262306 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:54.262866 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:54.262884 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:54.265522 (+ 2638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":268,"threads_started":1}
W20260618 22:33:54.265956 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:54.258238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:52747 (local address 127.18.150.190:42333)
0618 22:33:54.258423 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:54.258428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:54.258769 (+ 341us) server_negotiation.cc:407] Connection header received
0618 22:33:54.258898 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:54.258901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:54.258962 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:54.259041 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:54.259696 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.260443 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.261576 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.261995 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.262085 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:54.265692 (+ 3607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:52747: BlockingRecv error: recv got EOF from 127.18.150.129:52747 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 18 22:33:54 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:54 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:54.680734 28286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:54.672650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:59851)
0618 22:33:54.673130 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:54.673146 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:54.673277 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:54.673590 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:54.673594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:54.674200 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:54.674385 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:54.674394 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.675485 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.675490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:54.676617 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:54.676630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.677256 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.677260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:54.677371 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:54.677950 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:54.677968 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:54.680523 (+ 2555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":247,"threads_started":1}
W20260618 22:33:54.680962 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:54.672781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:59851 (local address 127.18.150.190:42333)
0618 22:33:54.672969 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:54.672975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:54.673226 (+ 251us) server_negotiation.cc:407] Connection header received
0618 22:33:54.673408 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:54.673412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:54.673472 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:54.673547 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:54.674534 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.675349 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.676740 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:54.677128 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:54.677226 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:54.680742 (+ 3516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:59851: BlockingRecv error: recv got EOF from 127.18.150.130:59851 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 18 22:33:55 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:55 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:55.109241 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:55.101015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:44051)
0618 22:33:55.101492 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:55.101507 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:55.101616 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:55.102011 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:55.102015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:55.102374 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:55.102544 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:55.102553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.103568 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.103572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:55.104485 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:55.104501 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.105413 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.105420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:55.105559 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:55.106253 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:55.106270 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:55.108940 (+ 2670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":265,"threads_started":1}
W20260618 22:33:55.109354 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:55.101105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:44051 (local address 127.18.150.190:42333)
0618 22:33:55.101271 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:55.101275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:55.101575 (+ 300us) server_negotiation.cc:407] Connection header received
0618 22:33:55.101751 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:55.101755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:55.101819 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:55.101905 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:55.102695 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.103438 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.104638 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.105247 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.105345 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:55.109123 (+ 3778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:44051: BlockingRecv error: recv got EOF from 127.18.150.131:44051 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 18 22:33:55 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:55 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:55.274868 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:55.266605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:54527)
0618 22:33:55.267153 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:55.267188 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:55.267302 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:55.267603 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:55.267607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:55.267998 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:55.268159 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:55.268167 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.269274 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.269278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:55.270327 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:55.270341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.271051 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.271056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:55.271189 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:55.271721 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:55.271738 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:55.274565 (+ 2827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":247,"threads_started":1}
W20260618 22:33:55.274942 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:55.266731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:54527 (local address 127.18.150.190:42333)
0618 22:33:55.266895 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:55.266899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:55.267281 (+ 382us) server_negotiation.cc:407] Connection header received
0618 22:33:55.267392 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:55.267396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:55.267462 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:55.267545 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:55.268339 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.269129 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.270470 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.270907 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.271004 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:55.274769 (+ 3765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:54527: BlockingRecv error: recv got EOF from 127.18.150.129:54527 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 18 22:33:55 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:55 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:55.689370 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:55.681600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:36891)
0618 22:33:55.682159 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:55.682175 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:55.682296 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:55.682589 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:55.682592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:55.682993 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:55.683150 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:55.683158 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.684269 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.684274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:55.685193 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:55.685205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.685849 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.685854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:55.686033 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:55.686566 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:55.686582 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:55.689107 (+ 2525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":273,"threads_started":1}
W20260618 22:33:55.689589 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:55.681730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:36891 (local address 127.18.150.190:42333)
0618 22:33:55.681888 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:55.681892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:55.682272 (+ 380us) server_negotiation.cc:407] Connection header received
0618 22:33:55.682390 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:55.682394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:55.682472 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:55.682565 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:55.683307 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.684109 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.685323 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:55.685714 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:55.685817 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:55.689343 (+ 3526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:36891: BlockingRecv error: recv got EOF from 127.18.150.130:36891 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 18 22:33:56 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:56 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:56.120561 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:56.110225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:54339)
0618 22:33:56.110767 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:56.110784 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:56.110925 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:56.111287 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:56.111291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:56.111788 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:56.112003 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:56.112017 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.113458 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.113464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:56.114936 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:56.114952 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.115875 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.115882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:56.116052 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:56.116725 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:56.116748 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:56.120174 (+ 3426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":242,"threads_started":1}
W20260618 22:33:56.120657 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:56.110353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:54339 (local address 127.18.150.190:42333)
0618 22:33:56.110557 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:56.110563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:56.110878 (+ 315us) server_negotiation.cc:407] Connection header received
0618 22:33:56.111053 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:56.111058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:56.111139 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:56.111229 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:56.112173 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.113303 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.115071 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.115717 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.115834 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:56.120419 (+ 4585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:54339: BlockingRecv error: recv got EOF from 127.18.150.131:54339 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Jun 18 22:33:56 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:56 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:56.283574 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:56.275746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:40983)
0618 22:33:56.276205 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:56.276224 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:56.276352 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:56.276641 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:56.276646 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:56.277044 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:56.277187 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:56.277196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.278186 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.278190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:56.279069 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:56.279079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.279796 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.279802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:56.279936 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:56.280485 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:56.280502 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:56.283270 (+ 2768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":231,"threads_started":1}
W20260618 22:33:56.283704 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:56.275884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:40983 (local address 127.18.150.190:42333)
0618 22:33:56.276075 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:56.276082 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:56.276291 (+ 209us) server_negotiation.cc:407] Connection header received
0618 22:33:56.276455 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:56.276461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:56.276522 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:56.276595 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:56.277311 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.278050 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.279189 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.279636 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.279737 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:56.283463 (+ 3726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:40983: BlockingRecv error: recv got EOF from 127.18.150.129:40983 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 18 22:33:56 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:56 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:56.698072 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:56.690227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:44063)
0618 22:33:56.690675 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:56.690691 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:56.690848 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:56.691209 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:56.691213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:56.691585 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:56.691735 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:56.691743 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.692927 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.692931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:56.693832 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:56.693842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.694530 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.694535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:56.694642 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:56.695191 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:56.695208 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:56.697736 (+ 2528us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":227,"threads_started":1}
W20260618 22:33:56.698161 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:56.690364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:44063 (local address 127.18.150.190:42333)
0618 22:33:56.690555 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:56.690561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:56.690779 (+ 218us) server_negotiation.cc:407] Connection header received
0618 22:33:56.691005 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:56.691009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:56.691074 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:56.691163 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:56.691891 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.692751 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.693967 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:56.694382 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:56.694469 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:56.697966 (+ 3497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:44063: BlockingRecv error: recv got EOF from 127.18.150.130:44063 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 18 22:33:57 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:57 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:57.129554 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:57.121663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:42151)
0618 22:33:57.122194 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:57.122212 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:57.122347 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:57.122669 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:57.122674 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:57.123095 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:57.123241 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:57.123250 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.124228 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.124233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:57.125177 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:57.125191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.125882 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.125889 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:57.126057 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:57.126630 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:57.126650 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:57.129289 (+ 2639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":290,"threads_started":1}
W20260618 22:33:57.129722 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:57.121754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:42151 (local address 127.18.150.190:42333)
0618 22:33:57.122016 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:57.122022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:57.122295 (+ 273us) server_negotiation.cc:407] Connection header received
0618 22:33:57.122460 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:57.122465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:57.122532 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:57.122630 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:57.123382 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.124086 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.125304 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.125725 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.125824 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:57.129511 (+ 3687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:42151: BlockingRecv error: recv got EOF from 127.18.150.131:42151 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 18 22:33:57 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:57 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:57.292799 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:57.284572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:45093)
0618 22:33:57.285069 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:57.285089 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:57.285212 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:57.285503 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:57.285507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:57.285877 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:57.286057 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:57.286067 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.287049 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.287053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:57.287968 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:57.287979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.288619 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.288624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:57.288784 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:57.289393 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:57.289415 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:57.292476 (+ 3061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":259,"threads_started":1}
W20260618 22:33:57.292886 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:57.284633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:45093 (local address 127.18.150.190:42333)
0618 22:33:57.284848 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:57.284854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:57.285175 (+ 321us) server_negotiation.cc:407] Connection header received
0618 22:33:57.285324 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:57.285328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:57.285386 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:57.285462 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:57.286192 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.286924 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.288093 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.288484 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.288599 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:57.292670 (+ 4071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:45093: BlockingRecv error: recv got EOF from 127.18.150.129:45093 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 18 22:33:57 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:57 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:57.706867 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:57.698956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:47683)
0618 22:33:57.699443 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:57.699459 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:57.699587 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:57.699913 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:57.699918 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:57.700320 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:57.700493 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:57.700503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.701469 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.701473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:57.702413 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:57.702424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.703071 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.703077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:57.703225 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:57.703783 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:57.703806 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:57.706639 (+ 2833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":266,"threads_started":1}
W20260618 22:33:57.707124 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:57.699012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:47683 (local address 127.18.150.190:42333)
0618 22:33:57.699176 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:57.699183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:57.699543 (+ 360us) server_negotiation.cc:407] Connection header received
0618 22:33:57.699697 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:57.699700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:57.699784 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:57.699864 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:57.700629 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.701341 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.702542 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:57.702941 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:57.703040 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:57.706874 (+ 3834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:47683: BlockingRecv error: recv got EOF from 127.18.150.130:47683 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 18 22:33:58 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:58 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:58.138597 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:58.130429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:45971)
0618 22:33:58.130929 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:58.130945 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:58.131076 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:58.131391 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:58.131394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:58.131740 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:58.131918 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:58.131926 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.132980 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.132985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:58.134103 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:58.134115 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.134812 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.134817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:58.134939 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:58.135511 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:58.135527 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:58.138257 (+ 2730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":264,"threads_started":1}
W20260618 22:33:58.138633 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:58.130515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:45971 (local address 127.18.150.190:42333)
0618 22:33:58.130682 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:58.130686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:58.131013 (+ 327us) server_negotiation.cc:407] Connection header received
0618 22:33:58.131205 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:58.131211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:58.131277 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:58.131365 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:58.132072 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.132848 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.134241 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.134659 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.134768 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:58.138430 (+ 3662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:45971: BlockingRecv error: recv got EOF from 127.18.150.131:45971 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 18 22:33:58 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:58 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:58.301270 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:58.293857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:37597)
0618 22:33:58.294341 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:58.294363 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:58.294476 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:58.294719 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:58.294721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:58.295046 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:58.295187 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:58.295195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.296167 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.296172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:58.297123 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:58.297137 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.297791 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.297795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:58.297903 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:58.298513 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:58.298531 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:58.301050 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":274,"threads_started":1}
W20260618 22:33:58.301443 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:58.293965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:37597 (local address 127.18.150.190:42333)
0618 22:33:58.294140 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:58.294145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:58.294430 (+ 285us) server_negotiation.cc:407] Connection header received
0618 22:33:58.294562 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:58.294565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:58.294623 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:58.294702 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:58.295297 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.296004 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.297238 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.297665 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.297749 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:58.301217 (+ 3468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:37597: BlockingRecv error: recv got EOF from 127.18.150.129:37597 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 18 22:33:58 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:58 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:58.715400 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:58.707728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:56305)
0618 22:33:58.708204 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:58.708218 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:58.708349 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:58.708635 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:58.708639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:58.708991 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:58.709123 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:58.709132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.710122 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.710127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:58.711044 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:58.711055 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.711705 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.711711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:58.711865 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:58.712404 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:58.712420 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:58.715108 (+ 2688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":250,"threads_started":1}
W20260618 22:33:58.715570 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:58.707818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:56305 (local address 127.18.150.190:42333)
0618 22:33:58.708003 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:58.708009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:58.708294 (+ 285us) server_negotiation.cc:407] Connection header received
0618 22:33:58.708455 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:58.708460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:58.708518 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:58.708594 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:58.709256 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.710013 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.711171 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:58.711572 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:58.711675 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:58.715299 (+ 3624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:56305: BlockingRecv error: recv got EOF from 127.18.150.130:56305 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 18 22:33:59 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:59 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:59.147748 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:59.139614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:48845)
0618 22:33:59.140111 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:59.140127 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:59.140271 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:59.140573 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:59.140578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:59.140936 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:59.141069 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:59.141078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.142347 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.142353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:59.143364 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:59.143378 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.144045 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.144051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:59.144210 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:59.144783 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:59.144801 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:59.147439 (+ 2638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":242,"threads_started":1}
W20260618 22:33:59.147895 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:59.139832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:48845 (local address 127.18.150.190:42333)
0618 22:33:59.140047 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:59.140054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:59.140199 (+ 145us) server_negotiation.cc:407] Connection header received
0618 22:33:59.140386 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:59.140391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:59.140453 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:59.140534 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:59.141210 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.142212 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.143499 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.143911 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.144015 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:59.147670 (+ 3655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:48845: BlockingRecv error: recv got EOF from 127.18.150.131:48845 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Jun 18 22:33:59 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:59 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:59.309547 28300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:59.302096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:47721)
0618 22:33:59.302552 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:59.302573 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:59.302689 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:59.302970 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:59.302973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:59.303352 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:59.303514 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:59.303522 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.304423 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.304426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:59.305294 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:59.305304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.305972 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.305978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:59.306107 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:59.306619 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:59.306637 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:59.309239 (+ 2602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":234,"threads_started":1}
W20260618 22:33:59.309640 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:59.302332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:47721 (local address 127.18.150.190:42333)
0618 22:33:59.302519 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:59.302525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:59.302627 (+ 102us) server_negotiation.cc:407] Connection header received
0618 22:33:59.302799 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:59.302803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:59.302862 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:59.302935 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:59.303618 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.304314 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.305390 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.305807 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.305911 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:59.309433 (+ 3522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:47721: BlockingRecv error: recv got EOF from 127.18.150.129:47721 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 18 22:33:59 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:33:59 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:33:59.724249 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:33:59.716292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:37779)
0618 22:33:59.716767 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0618 22:33:59.716785 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:33:59.716920 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:33:59.717220 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:33:59.717224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:33:59.717720 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:33:59.717895 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:59.717904 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.718929 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.718933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:59.719826 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:33:59.719837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.720668 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.720674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:33:59.720836 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:59.721376 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:33:59.721392 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:33:59.724016 (+ 2624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":241,"threads_started":1}
W20260618 22:33:59.724501 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:33:59.716400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:37779 (local address 127.18.150.190:42333)
0618 22:33:59.716578 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:33:59.716584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:33:59.716868 (+ 284us) server_negotiation.cc:407] Connection header received
0618 22:33:59.717035 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:33:59.717039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:33:59.717097 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:33:59.717172 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:33:59.718062 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.718813 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.719951 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:33:59.720537 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:33:59.720637 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:33:59.724257 (+ 3620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:37779: BlockingRecv error: recv got EOF from 127.18.150.130:37779 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 18 22:34:00 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:00 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:00.157258 28302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:00.148892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:46109)
0618 22:34:00.149374 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:00.149395 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:00.149534 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:00.149900 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:00.149906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:00.150401 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:00.150628 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:00.150637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.151651 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.151679 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:00.152583 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:00.152595 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.153284 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.153290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:00.153435 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:00.154137 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:00.154157 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:00.157028 (+ 2871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":304,"threads_started":1}
W20260618 22:34:00.157483 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:00.149031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:46109 (local address 127.18.150.190:42333)
0618 22:34:00.149200 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:00.149206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:00.149489 (+ 283us) server_negotiation.cc:407] Connection header received
0618 22:34:00.149646 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:00.149651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:00.149728 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:00.149827 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:00.150803 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.151523 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.152702 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.153096 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.153185 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:00.157234 (+ 4049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:46109: BlockingRecv error: recv got EOF from 127.18.150.131:46109 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 18 22:34:00 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:00 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:00.318248 28303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:00.310647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:55351)
0618 22:34:00.311091 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:00.311110 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:00.311217 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:00.311460 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:00.311463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:00.311828 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:00.311972 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:00.311980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.312921 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.312925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:00.313822 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:00.313833 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.314514 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.314518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:00.314648 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:00.315169 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:00.315184 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:00.317983 (+ 2799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":263,"threads_started":1}
W20260618 22:34:00.318401 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:00.310689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:55351 (local address 127.18.150.190:42333)
0618 22:34:00.310918 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:00.310926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:00.311178 (+ 252us) server_negotiation.cc:407] Connection header received
0618 22:34:00.311296 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:00.311299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:00.311357 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:00.311431 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:00.312094 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.312810 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.313956 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.314409 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.314498 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:00.318167 (+ 3669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:55351: BlockingRecv error: recv got EOF from 127.18.150.129:55351 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Jun 18 22:34:00 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:00 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:00.733388 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:00.725128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:50997)
0618 22:34:00.725612 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:00.725627 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:00.725750 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:00.726072 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:00.726077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:00.726528 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:00.726730 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:00.726740 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.727670 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.727676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:00.728581 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:00.728592 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.729226 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.729232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:00.729394 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:00.729965 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:00.729989 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:00.733075 (+ 3086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":258,"threads_started":1}
W20260618 22:34:00.733431 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:00.725207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:50997 (local address 127.18.150.190:42333)
0618 22:34:00.725383 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:00.725390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:00.725720 (+ 330us) server_negotiation.cc:407] Connection header received
0618 22:34:00.725877 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:00.725881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:00.725955 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:00.726031 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:00.726853 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.727545 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.728701 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:00.729096 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:00.729194 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:00.733257 (+ 4063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:50997: BlockingRecv error: recv got EOF from 127.18.150.130:50997 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 18 22:34:01 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:01 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:01.172914 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:01.158236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:43643)
0618 22:34:01.158748 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:01.158764 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:01.158882 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:01.159150 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:01.159153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:01.159523 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:01.159670 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:01.159678 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.160647 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.160651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:01.161665 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:01.161677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.162431 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.162437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:01.162621 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:01.169995 (+ 7374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:01.170023 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:01.172609 (+ 2586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":332,"threads_started":1}
W20260618 22:34:01.173105 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:01.158305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:43643 (local address 127.18.150.190:42333)
0618 22:34:01.158460 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:01.158464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:01.158846 (+ 382us) server_negotiation.cc:407] Connection header received
0618 22:34:01.158974 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:01.158978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:01.159042 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:01.159124 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:01.159816 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.160517 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.161807 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.162289 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.162412 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:01.172863 (+ 10451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:43643: BlockingRecv error: recv got EOF from 127.18.150.131:43643 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 18 22:34:01 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:01 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:01.326918 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:01.319106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:47935)
0618 22:34:01.319612 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:01.319630 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:01.319756 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:01.320039 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:01.320045 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:01.320419 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:01.320586 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:01.320600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.321528 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.321532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:01.322588 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:01.322600 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.323275 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.323281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:01.323418 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:01.323935 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:01.323953 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:01.326609 (+ 2656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":245,"threads_started":1}
W20260618 22:34:01.327067 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:01.319232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:47935 (local address 127.18.150.190:42333)
0618 22:34:01.319439 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:01.319445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:01.319705 (+ 260us) server_negotiation.cc:407] Connection header received
0618 22:34:01.319858 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:01.319863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:01.319920 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:01.319996 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:01.320718 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.321406 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.322714 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.323111 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.323219 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:01.326834 (+ 3615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:47935: BlockingRecv error: recv got EOF from 127.18.150.129:47935 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 18 22:34:01 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:01 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:01.742668 28307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:01.734281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:33019)
0618 22:34:01.734780 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:01.734795 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:01.734902 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:01.735191 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:01.735193 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:01.735549 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:01.735698 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:01.735706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.737087 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.737091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:01.738057 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:01.738070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.738886 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.738890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:01.739005 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:01.739499 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:01.739515 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:01.742362 (+ 2847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":242,"threads_started":1}
W20260618 22:34:01.742770 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:01.734420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:33019 (local address 127.18.150.190:42333)
0618 22:34:01.734586 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:01.734592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:01.734861 (+ 269us) server_negotiation.cc:407] Connection header received
0618 22:34:01.734992 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:01.734996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:01.735068 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:01.735181 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:01.735821 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.736965 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.738179 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:01.738742 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:01.738834 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:01.742566 (+ 3732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:33019: BlockingRecv error: recv got EOF from 127.18.150.130:33019 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 18 22:34:02 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:02 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:02.181449 28308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:02.173773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:57329)
0618 22:34:02.174287 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:02.174307 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:02.174435 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:02.174752 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:02.174757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:02.175145 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:02.175293 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:02.175302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.176243 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.176249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:02.177136 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:02.177147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.177796 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.177802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:02.177978 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:02.178572 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:02.178587 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:02.181171 (+ 2584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":264,"threads_started":1}
W20260618 22:34:02.181607 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:02.173873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:57329 (local address 127.18.150.190:42333)
0618 22:34:02.174058 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:02.174064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:02.174384 (+ 320us) server_negotiation.cc:407] Connection header received
0618 22:34:02.174554 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:02.174559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:02.174628 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:02.174705 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:02.175429 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.176113 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.177270 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.177669 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.177770 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:02.181392 (+ 3622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:57329: BlockingRecv error: recv got EOF from 127.18.150.131:57329 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 18 22:34:02 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:02 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:02.335376 28309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:02.327777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:43761)
0618 22:34:02.328191 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:02.328212 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:02.328318 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:02.328563 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:02.328567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:02.328913 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:02.329078 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:02.329086 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.330112 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.330116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:02.330990 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:02.331001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.331653 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.331657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:02.331767 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:02.332359 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:02.332377 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:02.335110 (+ 2733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":217,"threads_started":1}
W20260618 22:34:02.335579 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:02.327881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:43761 (local address 127.18.150.190:42333)
0618 22:34:02.328118 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:02.328122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:02.328271 (+ 149us) server_negotiation.cc:407] Connection header received
0618 22:34:02.328399 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:02.328402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:02.328461 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:02.328539 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:02.329197 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.330000 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.331129 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.331545 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.331646 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:02.335328 (+ 3682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:43761: BlockingRecv error: recv got EOF from 127.18.150.129:43761 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Jun 18 22:34:02 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:02 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:02.750706 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:02.743487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:51229)
0618 22:34:02.743881 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:02.743897 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:02.744018 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:02.744269 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:02.744272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:02.744605 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:02.744775 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:02.744783 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.745678 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.745682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:02.746618 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:02.746630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.747202 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.747206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:02.747330 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:02.747881 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:02.747897 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:02.750480 (+ 2583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":197,"threads_started":1}
W20260618 22:34:02.750808 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:02.743549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:51229 (local address 127.18.150.190:42333)
0618 22:34:02.743730 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:02.743736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:02.743972 (+ 236us) server_negotiation.cc:407] Connection header received
0618 22:34:02.744104 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:02.744107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:02.744166 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:02.744243 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:02.744872 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.745575 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.746714 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:02.747104 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:02.747187 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:02.750638 (+ 3451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:51229: BlockingRecv error: recv got EOF from 127.18.150.130:51229 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 18 22:34:03 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:03 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.131@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:03.190353 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:03.182435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.131:59325)
0618 22:34:03.183075 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:03.183096 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:03.183231 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:03.183598 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:03.183603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:03.184031 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:03.184215 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:03.184224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.185153 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.185157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:03.186060 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:03.186070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.186747 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.186751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:03.186876 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:03.187419 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:03.187435 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:03.190062 (+ 2627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":428,"threads_started":1}
W20260618 22:34:03.190560 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:03.182509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:59325 (local address 127.18.150.190:42333)
0618 22:34:03.182722 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:03.182729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:03.183178 (+ 449us) server_negotiation.cc:407] Connection header received
0618 22:34:03.183358 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:03.183363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:03.183451 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:03.183550 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:03.184350 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.185041 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.186187 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.186629 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.186723 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:03.190360 (+ 3637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.131:59325: BlockingRecv error: recv got EOF from 127.18.150.131:59325 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 18 22:34:03 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:03 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.129@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:03.344744 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:03.336454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.129:43795)
0618 22:34:03.337022 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:03.337042 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:03.337158 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:03.337460 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:03.337463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:03.337858 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:03.338045 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:03.338053 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.339071 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.339075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:03.340181 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:03.340194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.341059 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.341065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:03.341203 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:03.341742 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:03.341760 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:03.344412 (+ 2652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":327,"threads_started":1}
W20260618 22:34:03.344861 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:03.336518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:43795 (local address 127.18.150.190:42333)
0618 22:34:03.336696 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:03.336700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:03.337111 (+ 411us) server_negotiation.cc:407] Connection header received
0618 22:34:03.337261 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:03.337265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:03.337326 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:03.337412 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:03.338192 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.338916 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.340308 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.340922 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.341042 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:03.344622 (+ 3580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.129:43795: BlockingRecv error: recv got EOF from 127.18.150.129:43795 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 18 22:34:03 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:34:03 dist-test-slave-h7jz krb5kdc[27268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.150.130@KRBTEST.COM for oryx/127.18.150.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:34:03.759888 28313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:03.751772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:42333 (local address 127.18.150.130:47549)
0618 22:34:03.752253 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:03.752268 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:03.752406 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:03.752700 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:03.752704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:03.753091 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:03.753248 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:03.753256 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.754282 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.754288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:03.755212 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:03.755224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.755905 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.755911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:03.756065 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:03.756672 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:03.756693 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:03.759627 (+ 2934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.150.190:42333: Server oryx/127.18.150.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":281,"threads_started":1}
W20260618 22:34:03.760085 28005 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:03.751912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:47549 (local address 127.18.150.190:42333)
0618 22:34:03.752100 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:03.752107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:03.752347 (+ 240us) server_negotiation.cc:407] Connection header received
0618 22:34:03.752518 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:03.752523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:03.752584 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:03.752661 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:03.753385 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.754149 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.755338 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:03.755774 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:03.755886 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:03.759865 (+ 3979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.150.130:47549: BlockingRecv error: recv got EOF from 127.18.150.130:47549 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:34:04.030553 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 27855
I20260618 22:34:04.041616 28315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:04.031483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33154 (local address 127.18.150.129:44067)
0618 22:34:04.031956 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:04.031963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:04.031990 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:34:04.032093 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:04.032102 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:04.032396 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:04.032557 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:04.033886 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:04.034903 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:04.036091 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:04.036617 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:04.036708 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:04.038943 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:04.038951 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:04.038965 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:04.038996 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:04.041092 (+ 2096us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.150.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0618 22:34:04.041333 (+ 241us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33154: Request ticket server kudu/127.18.150.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":228,"threads_started":1}
W20260618 22:34:04.041711 28315 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33154: Request ticket server kudu/127.18.150.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260618 22:34:04.041705 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 27855: Not authorized: Client connection negotiation failed: client connection to 127.18.150.129:44067: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.150.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260618 22:34:04.041985 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 27855
I20260618 22:34:04.054955 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 28006
I20260618 22:34:04.066149 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:04.055943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.18.150.130:36243)
0618 22:34:04.056498 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:04.056505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:04.056534 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:34:04.056590 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:04.056596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:04.056804 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:04.056955 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:04.058248 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:04.059271 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:04.060444 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:04.060965 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:04.061051 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:04.063335 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:04.063343 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:04.063357 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:04.063384 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:04.065631 (+ 2247us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.150.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0618 22:34:04.065864 (+ 233us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34310: Request ticket server kudu/127.18.150.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":266,"threads_started":1}
W20260618 22:34:04.066243 28316 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34310: Request ticket server kudu/127.18.150.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260618 22:34:04.066231 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 28006: Not authorized: Client connection negotiation failed: client connection to 127.18.150.130:36243: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.150.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260618 22:34:04.066390 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 28006
I20260618 22:34:04.079227 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 28137
I20260618 22:34:04.090468 28317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:04.080103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.18.150.131:37215)
0618 22:34:04.080653 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:04.080659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:04.080692 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:34:04.080749 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:04.080755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:04.080962 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:04.081103 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:04.082318 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:04.083379 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:04.084618 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:04.085169 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:04.085270 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:04.087592 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:04.087600 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:04.087614 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:04.087644 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:04.089927 (+ 2283us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.150.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0618 22:34:04.090203 (+ 276us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46954: Request ticket server kudu/127.18.150.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":267,"threads_started":1}
W20260618 22:34:04.090554 28317 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46954: Request ticket server kudu/127.18.150.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260618 22:34:04.090584 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 28137: Not authorized: Client connection negotiation failed: client connection to 127.18.150.131:37215: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.150.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260618 22:34:04.090765 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 28137
I20260618 22:34:04.103295 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 27786
I20260618 22:34:04.104465 27848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:04.215023 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 27786
2026-06-18T22:34:04Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20699 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-18T22:34:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:34:04Z Disabled control of system clock
I20260618 22:34:04.280558 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:46485
--webserver_interface=127.18.150.190
--webserver_port=0
--builtin_ntp_servers=127.18.150.148:46489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:46485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260618 22:34:04.588523 28322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:04.588857 28322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:04.588936 28322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:04.596341 28322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:34:04.596454 28322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:04.596490 28322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:04.596536 28322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:34:04.596578 28322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:34:04.605731 28322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:46485
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:46485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:04.607041 28322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:04.608717 28322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:04.619567 28328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:04.619544 28330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:04.619623 28327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:04.620289 28322 server_base.cc:1034] running on GCE node
I20260618 22:34:04.621011 28322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:04.622388 28322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:04.623576 28322 hybrid_clock.cc:648] HybridClock initialized: now 1781822044623527 us; error 54 us; skew 500 ppm
I20260618 22:34:04.623886 28322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:34:04.625960 28322 webserver.cc:460] Webserver started at http://127.18.150.190:40277/ using document root <none> and password file <none>
I20260618 22:34:04.626631 28322 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:04.626724 28322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:04.627003 28322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:04.629683 28322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "071fd13e4f8647b3911188e124bc7a08"
format_stamp: "Formatted at 2026-06-18 22:34:04 on dist-test-slave-h7jz"
server_key: "7aab4ff5d33a59536334990d593ef679"
server_key_iv: "9f03533cf71bb8c52387b9d9c2f45099"
server_key_version: "encryptionkey@0"
I20260618 22:34:04.630441 28322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "071fd13e4f8647b3911188e124bc7a08"
format_stamp: "Formatted at 2026-06-18 22:34:04 on dist-test-slave-h7jz"
server_key: "7aab4ff5d33a59536334990d593ef679"
server_key_iv: "9f03533cf71bb8c52387b9d9c2f45099"
server_key_version: "encryptionkey@0"
I20260618 22:34:04.635267 28322 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260618 22:34:04.638430 28336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:04.639743 28322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:34:04.639899 28322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "071fd13e4f8647b3911188e124bc7a08"
format_stamp: "Formatted at 2026-06-18 22:34:04 on dist-test-slave-h7jz"
server_key: "7aab4ff5d33a59536334990d593ef679"
server_key_iv: "9f03533cf71bb8c52387b9d9c2f45099"
server_key_version: "encryptionkey@0"
I20260618 22:34:04.640077 28322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:04.666239 28322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:04.667063 28322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:04.667326 28322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:04.683670 28322 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:46485
I20260618 22:34:04.683689 28387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:46485 every 8 connection(s)
I20260618 22:34:04.684993 28322 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:34:04.687780 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 28322
I20260618 22:34:04.687965 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:34:04.688742 19034 external_mini_cluster.cc:1428] Setting key 508165dff9107379491eb3277314dc53
I20260618 22:34:04.689242 28388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:34:04.701363 28388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08: Bootstrap starting.
I20260618 22:34:04.702133 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:04.690757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58218 (local address 127.18.150.190:46485)
0618 22:34:04.691730 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:04.691742 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:04.691787 (+ 45us) server_negotiation.cc:407] Connection header received
0618 22:34:04.693769 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:04.693791 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:04.694308 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:04.694698 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:04.695266 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:04.696534 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:04.697836 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:04.698573 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:04.698747 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:04.699838 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:04.699846 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:04.699860 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:04.700373 (+ 513us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:04.700450 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:04.700470 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:04.700563 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:04.700774 (+ 211us) server_negotiation.cc:299] Negotiation successful
0618 22:34:04.701071 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":358,"threads_started":1}
I20260618 22:34:04.704437 28388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08: Neither blocks nor log segments found. Creating new log.
I20260618 22:34:04.705456 28388 log.cc:824] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08: Log is configured to *not* fsync() on all Append() calls
I20260618 22:34:04.708125 28388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08: No bootstrap required, opened a new log
I20260618 22:34:04.716512 28388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071fd13e4f8647b3911188e124bc7a08" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46485 } }
I20260618 22:34:04.716900 28388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:34:04.716984 28388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 071fd13e4f8647b3911188e124bc7a08, State: Initialized, Role: FOLLOWER
I20260618 22:34:04.717566 28388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071fd13e4f8647b3911188e124bc7a08" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46485 } }
I20260618 22:34:04.717785 28388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:34:04.717896 28388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:34:04.718082 28388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:34:04.720281 28388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071fd13e4f8647b3911188e124bc7a08" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46485 } }
I20260618 22:34:04.720762 28388 leader_election.cc:302] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 071fd13e4f8647b3911188e124bc7a08; no voters:
I20260618 22:34:04.721308 28388 leader_election.cc:288] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:34:04.721429 28393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:34:04.722059 28393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [term 1 LEADER]: Becoming Leader. State: Replica: 071fd13e4f8647b3911188e124bc7a08, State: Running, Role: LEADER
I20260618 22:34:04.722595 28393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071fd13e4f8647b3911188e124bc7a08" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46485 } }
I20260618 22:34:04.723490 28388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:34:04.725301 28394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "071fd13e4f8647b3911188e124bc7a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071fd13e4f8647b3911188e124bc7a08" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46485 } } }
I20260618 22:34:04.725682 28394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [sys.catalog]: This master's current role is: LEADER
I20260618 22:34:04.726912 28395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 071fd13e4f8647b3911188e124bc7a08. Latest consensus state: current_term: 1 leader_uuid: "071fd13e4f8647b3911188e124bc7a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071fd13e4f8647b3911188e124bc7a08" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46485 } } }
I20260618 22:34:04.727295 28395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08 [sys.catalog]: This master's current role is: LEADER
I20260618 22:34:04.727626 28401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:34:04.733525 28401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:34:04.742697 28401 catalog_manager.cc:1347] Generated new cluster ID: 59149f28ba654185ad08f0758fdf0a3f
I20260618 22:34:04.742808 28401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:34:04.758117 28401 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:34:04.759444 28401 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:34:04.769306 28401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 071fd13e4f8647b3911188e124bc7a08: Generated new TSK 0
I20260618 22:34:04.770150 28401 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:34:04.774843 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46485
--builtin_ntp_servers=127.18.150.148:46489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260618 22:34:05.113806 28412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:05.114107 28412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:05.114205 28412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:05.121233 28412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:05.121320 28412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:05.121418 28412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:34:05.130376 28412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:05.131549 28412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:05.133177 28412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:05.145201 28420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:05.145232 28417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:05.145435 28418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:05.146190 28412 server_base.cc:1034] running on GCE node
I20260618 22:34:05.146716 28412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:05.147553 28412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:05.148747 28412 hybrid_clock.cc:648] HybridClock initialized: now 1781822045148684 us; error 59 us; skew 500 ppm
I20260618 22:34:05.149035 28412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:34:05.151152 28412 webserver.cc:460] Webserver started at http://127.18.150.129:35821/ using document root <none> and password file <none>
I20260618 22:34:05.151799 28412 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:05.151901 28412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:05.152189 28412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:05.154999 28412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "869bcc15687a47659e6b3c7a08313029"
format_stamp: "Formatted at 2026-06-18 22:34:05 on dist-test-slave-h7jz"
server_key: "ba5d3c8478017f32287b8081b69d3409"
server_key_iv: "75c8d75bde8f29dc7528f558ad07ff61"
server_key_version: "encryptionkey@0"
I20260618 22:34:05.155715 28412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "869bcc15687a47659e6b3c7a08313029"
format_stamp: "Formatted at 2026-06-18 22:34:05 on dist-test-slave-h7jz"
server_key: "ba5d3c8478017f32287b8081b69d3409"
server_key_iv: "75c8d75bde8f29dc7528f558ad07ff61"
server_key_version: "encryptionkey@0"
I20260618 22:34:05.160451 28412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:34:05.163586 28426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:05.164795 28412 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:34:05.164937 28412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "869bcc15687a47659e6b3c7a08313029"
format_stamp: "Formatted at 2026-06-18 22:34:05 on dist-test-slave-h7jz"
server_key: "ba5d3c8478017f32287b8081b69d3409"
server_key_iv: "75c8d75bde8f29dc7528f558ad07ff61"
server_key_version: "encryptionkey@0"
I20260618 22:34:05.165113 28412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:05.191649 28412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:05.192421 28412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:05.192685 28412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:05.193526 28412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:05.194840 28412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:05.194929 28412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:05.195026 28412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:05.195093 28412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:05.222677 28412 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:35593
I20260618 22:34:05.222729 28538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:35593 every 8 connection(s)
I20260618 22:34:05.223948 28412 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:34:05.228820 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 28412
I20260618 22:34:05.228978 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:34:05.229550 19034 external_mini_cluster.cc:1428] Setting key 907716ae522b55180251aaab9cb71e23
I20260618 22:34:05.232203 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46485
--builtin_ntp_servers=127.18.150.148:46489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260618 22:34:05.236270 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:05.226290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:58915 (local address 127.18.150.190:46485)
0618 22:34:05.226675 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:05.226679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:05.227927 (+ 1248us) server_negotiation.cc:407] Connection header received
0618 22:34:05.229696 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:05.229701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:05.229782 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:05.229897 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:05.230848 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:05.231685 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:05.233790 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:05.234206 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:05.234306 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:05.235606 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:05.235610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:05.235614 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:05.235630 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:05.235673 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:05.235684 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:05.235774 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:05.235962 (+ 188us) server_negotiation.cc:299] Negotiation successful
0618 22:34:05.236034 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":207,"threads_started":1}
I20260618 22:34:05.237478 28542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:05.226861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46485 (local address 127.18.150.129:58915)
0618 22:34:05.227786 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:05.227829 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:05.229441 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:05.230095 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:05.230103 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:05.230190 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:05.230706 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:05.230722 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:05.231837 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:05.231843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:05.233655 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:05.233669 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:05.234411 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:05.234416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:05.234661 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:05.235294 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:34:05.235297 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:05.235350 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:34:05.235440 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:05.235764 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:05.235772 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:05.235778 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:34:05.235945 (+ 167us) client_negotiation.cc:240] Negotiation successful
0618 22:34:05.236281 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":232,"threads_started":1}
I20260618 22:34:05.238799 28539 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46485
I20260618 22:34:05.239058 28539 heartbeater.cc:464] Registering TS with master...
I20260618 22:34:05.239774 28539 heartbeater.cc:511] Master 127.18.150.190:46485 requested a full tablet report, sending...
I20260618 22:34:05.241648 28353 ts_manager.cc:194] Registered new tserver with Master: 869bcc15687a47659e6b3c7a08313029 (127.18.150.129:35593)
I20260618 22:34:05.243083 28353 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.129:58915
W20260618 22:34:05.540179 28543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:05.540441 28543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:05.540539 28543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:05.547608 28543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:05.547701 28543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:05.547838 28543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:34:05.556658 28543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:05.557906 28543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:05.559604 28543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:05.571427 28549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:05.571655 28548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:05.571890 28551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:05.572229 28543 server_base.cc:1034] running on GCE node
I20260618 22:34:05.572726 28543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:05.573539 28543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:05.574716 28543 hybrid_clock.cc:648] HybridClock initialized: now 1781822045574674 us; error 35 us; skew 500 ppm
I20260618 22:34:05.574991 28543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:34:05.576892 28543 webserver.cc:460] Webserver started at http://127.18.150.130:33687/ using document root <none> and password file <none>
I20260618 22:34:05.577497 28543 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:05.577575 28543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:05.577849 28543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:05.580437 28543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "c6aaa22c14bf444696b5dc19f6f710fe"
format_stamp: "Formatted at 2026-06-18 22:34:05 on dist-test-slave-h7jz"
server_key: "28f066327155fc758f546eedcf8f4337"
server_key_iv: "b21f17dfd4b5ea88ce54c14ee0c2ef91"
server_key_version: "encryptionkey@0"
I20260618 22:34:05.581131 28543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "c6aaa22c14bf444696b5dc19f6f710fe"
format_stamp: "Formatted at 2026-06-18 22:34:05 on dist-test-slave-h7jz"
server_key: "28f066327155fc758f546eedcf8f4337"
server_key_iv: "b21f17dfd4b5ea88ce54c14ee0c2ef91"
server_key_version: "encryptionkey@0"
I20260618 22:34:05.585843 28543 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260618 22:34:05.589058 28557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:05.590284 28543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:34:05.590426 28543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "c6aaa22c14bf444696b5dc19f6f710fe"
format_stamp: "Formatted at 2026-06-18 22:34:05 on dist-test-slave-h7jz"
server_key: "28f066327155fc758f546eedcf8f4337"
server_key_iv: "b21f17dfd4b5ea88ce54c14ee0c2ef91"
server_key_version: "encryptionkey@0"
I20260618 22:34:05.590606 28543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:05.603152 28543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:05.603914 28543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:05.604173 28543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:05.605033 28543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:05.606369 28543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:05.606458 28543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:05.606564 28543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:05.606631 28543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:05.634426 28543 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:45315
I20260618 22:34:05.634473 28669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:45315 every 8 connection(s)
I20260618 22:34:05.635702 28543 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:34:05.643891 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 28543
I20260618 22:34:05.644061 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:34:05.644558 19034 external_mini_cluster.cc:1428] Setting key 02da4c185b7fd65fa57e44c7e5a5691d
I20260618 22:34:05.647123 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46485
--builtin_ntp_servers=127.18.150.148:46489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260618 22:34:05.647536 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:05.638052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:39899 (local address 127.18.150.190:46485)
0618 22:34:05.638211 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:05.638216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:05.639536 (+ 1320us) server_negotiation.cc:407] Connection header received
0618 22:34:05.641156 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:05.641160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:05.641222 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:05.641303 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:05.642515 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:05.643289 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:05.644866 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:05.645311 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:05.645411 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:05.646813 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:05.646818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:05.646821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:05.646836 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:05.646865 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:05.646876 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:05.646947 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:05.647156 (+ 209us) server_negotiation.cc:299] Negotiation successful
0618 22:34:05.647241 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:34:05.648487 28672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:05.638580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46485 (local address 127.18.150.130:39899)
0618 22:34:05.639418 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:05.639452 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:05.640939 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:05.641526 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:05.641535 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:05.641620 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:05.642364 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:05.642384 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:05.643411 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:05.643414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:05.644728 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:05.644739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:05.645518 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:05.645525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:05.645806 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:05.646533 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:34:05.646535 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:05.646574 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:34:05.646646 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:05.646956 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:05.646966 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:05.646974 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:34:05.647119 (+ 145us) client_negotiation.cc:240] Negotiation successful
0618 22:34:05.647501 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":246,"threads_started":1}
I20260618 22:34:05.649798 28670 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46485
I20260618 22:34:05.650077 28670 heartbeater.cc:464] Registering TS with master...
I20260618 22:34:05.650802 28670 heartbeater.cc:511] Master 127.18.150.190:46485 requested a full tablet report, sending...
I20260618 22:34:05.652040 28353 ts_manager.cc:194] Registered new tserver with Master: c6aaa22c14bf444696b5dc19f6f710fe (127.18.150.130:45315)
I20260618 22:34:05.652782 28353 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.130:39899
W20260618 22:34:05.966010 28673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:05.966271 28673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:05.966349 28673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:05.973426 28673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:05.973520 28673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:05.973619 28673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:34:05.982353 28673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:46489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:05.983507 28673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:05.985235 28673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:05.997435 28678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:05.997493 28679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:05.997617 28681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:05.998637 28673 server_base.cc:1034] running on GCE node
I20260618 22:34:05.999272 28673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:06.000253 28673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:06.001453 28673 hybrid_clock.cc:648] HybridClock initialized: now 1781822046001390 us; error 50 us; skew 500 ppm
I20260618 22:34:06.001762 28673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:34:06.003935 28673 webserver.cc:460] Webserver started at http://127.18.150.131:43119/ using document root <none> and password file <none>
I20260618 22:34:06.004599 28673 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:06.004683 28673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:06.004954 28673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:06.007682 28673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "4b0b90a4f8ba439b8f4af7c7c3100023"
format_stamp: "Formatted at 2026-06-18 22:34:05 on dist-test-slave-h7jz"
server_key: "4314736ed72050dcb8ff73e3ef166f08"
server_key_iv: "eeeaf493fab847c648d7690b80353aa6"
server_key_version: "encryptionkey@0"
I20260618 22:34:06.008397 28673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "4b0b90a4f8ba439b8f4af7c7c3100023"
format_stamp: "Formatted at 2026-06-18 22:34:05 on dist-test-slave-h7jz"
server_key: "4314736ed72050dcb8ff73e3ef166f08"
server_key_iv: "eeeaf493fab847c648d7690b80353aa6"
server_key_version: "encryptionkey@0"
I20260618 22:34:06.013252 28673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:34:06.016584 28687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:06.017866 28673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:34:06.018049 28673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "4b0b90a4f8ba439b8f4af7c7c3100023"
format_stamp: "Formatted at 2026-06-18 22:34:05 on dist-test-slave-h7jz"
server_key: "4314736ed72050dcb8ff73e3ef166f08"
server_key_iv: "eeeaf493fab847c648d7690b80353aa6"
server_key_version: "encryptionkey@0"
I20260618 22:34:06.018217 28673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:06.029285 28673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:06.030027 28673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:06.030262 28673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:06.031090 28673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:06.032321 28673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:06.032399 28673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:06.032470 28673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:06.032522 28673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:06.060559 28673 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:36765
I20260618 22:34:06.060592 28799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:36765 every 8 connection(s)
I20260618 22:34:06.061733 28673 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:34:06.067111 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 28673
I20260618 22:34:06.067248 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:34:06.067646 19034 external_mini_cluster.cc:1428] Setting key 693e5944fd0a7af692d559c9c53c4522
I20260618 22:34:06.073354 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:06.064023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:41041 (local address 127.18.150.190:46485)
0618 22:34:06.064188 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:06.064192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:06.065527 (+ 1335us) server_negotiation.cc:407] Connection header received
0618 22:34:06.067215 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:06.067220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:06.067310 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:06.067406 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:06.068490 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:06.069340 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:06.070945 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:06.071352 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:06.071455 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:06.072597 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:06.072602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:06.072607 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:06.072623 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:06.072651 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:06.072661 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:06.072731 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:06.072957 (+ 226us) server_negotiation.cc:299] Negotiation successful
0618 22:34:06.073045 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:34:06.074304 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:06.064480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46485 (local address 127.18.150.131:41041)
0618 22:34:06.065381 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:06.065424 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:06.066957 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:06.067626 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:06.067634 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:06.067729 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:06.068325 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:06.068343 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:06.069479 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:06.069485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:06.070817 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:06.070829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:06.071557 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:06.071564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:06.071769 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:06.072321 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:34:06.072324 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:06.072363 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:34:06.072430 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:06.072741 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:06.072778 (+ 37us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:06.072785 (+ 7us) client_negotiation.cc:769] Sending connection context
0618 22:34:06.072939 (+ 154us) client_negotiation.cc:240] Negotiation successful
0618 22:34:06.073292 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":253,"threads_started":1}
I20260618 22:34:06.075404 28800 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46485
I20260618 22:34:06.075647 28800 heartbeater.cc:464] Registering TS with master...
I20260618 22:34:06.076326 28800 heartbeater.cc:511] Master 127.18.150.190:46485 requested a full tablet report, sending...
I20260618 22:34:06.077538 28353 ts_manager.cc:194] Registered new tserver with Master: 4b0b90a4f8ba439b8f4af7c7c3100023 (127.18.150.131:36765)
I20260618 22:34:06.078297 28353 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.131:41041
I20260618 22:34:06.082245 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260618 22:34:06.088107 28541 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:06.086620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58232 (local address 127.18.150.190:46485)
0618 22:34:06.086812 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:06.086817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:06.087270 (+ 453us) server_negotiation.cc:407] Connection header received
0618 22:34:06.087384 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:06.087388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:06.087457 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:06.087531 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:06.087921 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58232: BlockingRecv error: recv got EOF from 127.0.0.1:58232 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:34:06.088441 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 28412
I20260618 22:34:06.095690 28809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:06.089207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44938 (local address 127.18.150.129:35593)
0618 22:34:06.089817 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:06.089825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:06.089869 (+ 44us) server_negotiation.cc:407] Connection header received
0618 22:34:06.090161 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:06.090171 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:06.090423 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:06.090540 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:06.090896 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:06.091800 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:06.092939 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:06.093518 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:06.093603 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:06.094498 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:06.094508 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:06.094525 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:06.095045 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:06.095088 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:06.095106 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:06.095182 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:06.095326 (+ 144us) server_negotiation.cc:299] Negotiation successful
0618 22:34:06.095425 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":367,"threads_started":1}
I20260618 22:34:06.096772 28534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:06.188083 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 28412
I20260618 22:34:06.200728 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 28543
I20260618 22:34:06.212287 28811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:06.201513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36706 (local address 127.18.150.130:45315)
0618 22:34:06.202016 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:06.202023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:06.202050 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:34:06.202110 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:06.202116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:06.202328 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:06.202444 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:06.202776 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:06.203717 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:06.204850 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:06.205377 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:06.205469 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:06.211034 (+ 5565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:06.211043 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:06.211057 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:06.211629 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:06.211677 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:06.211697 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:06.211787 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:06.211887 (+ 100us) server_negotiation.cc:299] Negotiation successful
0618 22:34:06.211995 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":215,"threads_started":1}
I20260618 22:34:06.213485 28665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:06.290925 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 28543
I20260618 22:34:06.303500 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 28673
I20260618 22:34:06.310873 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:06.304329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51154 (local address 127.18.150.131:36765)
0618 22:34:06.304956 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:06.304963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:06.304990 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:34:06.305047 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:06.305054 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:06.305263 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:06.305384 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:06.305759 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:06.306811 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:06.307991 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:06.308538 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:06.308634 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:06.309497 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:06.309505 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:06.309517 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:06.310160 (+ 643us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:06.310205 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:06.310224 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:06.310317 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:06.310398 (+ 81us) server_negotiation.cc:299] Negotiation successful
0618 22:34:06.310512 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":322,"threads_started":1}
I20260618 22:34:06.312022 28795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:06.390290 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 28673
I20260618 22:34:06.402694 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 28322
I20260618 22:34:06.403805 28383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:06.502959 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 28322
2026-06-18T22:34:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2283 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-18T22:34:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:34:06Z Disabled control of system clock
I20260618 22:34:06.564043 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39481
--webserver_interface=127.18.150.190
--webserver_port=0
--builtin_ntp_servers=127.18.150.148:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:39481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260618 22:34:06.870409 28819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:06.870687 28819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:06.870785 28819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:06.877811 28819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:34:06.877916 28819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:06.877986 28819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:06.878051 28819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:34:06.878094 28819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:34:06.887040 28819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:39481
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:06.888217 28819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:06.889858 28819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:06.900502 28824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:06.900467 28825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:06.900523 28827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:06.901067 28819 server_base.cc:1034] running on GCE node
I20260618 22:34:06.901846 28819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:06.903259 28819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:06.904460 28819 hybrid_clock.cc:648] HybridClock initialized: now 1781822046904415 us; error 41 us; skew 500 ppm
I20260618 22:34:06.904769 28819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:34:06.906929 28819 webserver.cc:460] Webserver started at http://127.18.150.190:33897/ using document root <none> and password file <none>
I20260618 22:34:06.907593 28819 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:06.907687 28819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:06.907984 28819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:06.910780 28819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "1a6fb84c034748b8bcc687acd953df2f"
format_stamp: "Formatted at 2026-06-18 22:34:06 on dist-test-slave-h7jz"
server_key: "e44be4c15285468e864ba3e5e229a1a6"
server_key_iv: "18c11b3bda6476f8f97bba6ca3c7de06"
server_key_version: "encryptionkey@0"
I20260618 22:34:06.911511 28819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "1a6fb84c034748b8bcc687acd953df2f"
format_stamp: "Formatted at 2026-06-18 22:34:06 on dist-test-slave-h7jz"
server_key: "e44be4c15285468e864ba3e5e229a1a6"
server_key_iv: "18c11b3bda6476f8f97bba6ca3c7de06"
server_key_version: "encryptionkey@0"
I20260618 22:34:06.916316 28819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:34:06.919541 28833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:06.920836 28819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:34:06.920979 28819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "1a6fb84c034748b8bcc687acd953df2f"
format_stamp: "Formatted at 2026-06-18 22:34:06 on dist-test-slave-h7jz"
server_key: "e44be4c15285468e864ba3e5e229a1a6"
server_key_iv: "18c11b3bda6476f8f97bba6ca3c7de06"
server_key_version: "encryptionkey@0"
I20260618 22:34:06.921154 28819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:06.926262 28819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:06.926945 28819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:06.927189 28819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:06.942844 28819 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:39481
I20260618 22:34:06.942868 28884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:39481 every 8 connection(s)
I20260618 22:34:06.944336 28819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:34:06.948001 28885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:34:06.951251 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 28819
I20260618 22:34:06.951457 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:34:06.952242 19034 external_mini_cluster.cc:1428] Setting key ce61ceeb78af6ca4ac6189cfc8038b8c
I20260618 22:34:06.959903 28885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f: Bootstrap starting.
I20260618 22:34:06.959905 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:06.954214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59740 (local address 127.18.150.190:39481)
0618 22:34:06.955108 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:06.955118 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:06.955154 (+ 36us) server_negotiation.cc:407] Connection header received
0618 22:34:06.956605 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:06.956627 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:06.957042 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:06.957397 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:06.958411 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:06.958417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:06.958431 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:06.958444 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:06.958499 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:06.958513 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:06.958704 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:06.958840 (+ 136us) server_negotiation.cc:299] Negotiation successful
0618 22:34:06.959059 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":300,"threads_started":1}
I20260618 22:34:06.963210 28885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f: Neither blocks nor log segments found. Creating new log.
I20260618 22:34:06.964099 28885 log.cc:824] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:34:06.966514 28885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f: No bootstrap required, opened a new log
I20260618 22:34:06.973364 28885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6fb84c034748b8bcc687acd953df2f" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39481 } }
I20260618 22:34:06.973737 28885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:34:06.973837 28885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a6fb84c034748b8bcc687acd953df2f, State: Initialized, Role: FOLLOWER
I20260618 22:34:06.974412 28885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6fb84c034748b8bcc687acd953df2f" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39481 } }
I20260618 22:34:06.974604 28885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:34:06.974720 28885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:34:06.974856 28885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:34:06.976784 28885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6fb84c034748b8bcc687acd953df2f" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39481 } }
I20260618 22:34:06.977224 28885 leader_election.cc:302] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a6fb84c034748b8bcc687acd953df2f; no voters:
I20260618 22:34:06.977722 28885 leader_election.cc:288] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:34:06.977838 28890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:34:06.978459 28890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [term 1 LEADER]: Becoming Leader. State: Replica: 1a6fb84c034748b8bcc687acd953df2f, State: Running, Role: LEADER
I20260618 22:34:06.978942 28890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6fb84c034748b8bcc687acd953df2f" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39481 } }
I20260618 22:34:06.979628 28885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:34:06.981256 28891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a6fb84c034748b8bcc687acd953df2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6fb84c034748b8bcc687acd953df2f" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39481 } } }
I20260618 22:34:06.981230 28892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a6fb84c034748b8bcc687acd953df2f. Latest consensus state: current_term: 1 leader_uuid: "1a6fb84c034748b8bcc687acd953df2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6fb84c034748b8bcc687acd953df2f" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39481 } } }
I20260618 22:34:06.981565 28892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [sys.catalog]: This master's current role is: LEADER
I20260618 22:34:06.981570 28891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f [sys.catalog]: This master's current role is: LEADER
I20260618 22:34:06.989702 28897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:34:06.993456 28897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:34:07.000540 28897 catalog_manager.cc:1347] Generated new cluster ID: 03c18f51994f4d9db0c8643309c2a660
I20260618 22:34:07.000645 28897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:34:07.034966 28897 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:34:07.035466 28897 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:34:07.045382 28897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a6fb84c034748b8bcc687acd953df2f: Generated new TSK 0
I20260618 22:34:07.046401 28897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:34:07.059043 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39481
--builtin_ntp_servers=127.18.150.148:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260618 22:34:07.368469 28909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:07.368741 28909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:07.368868 28909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:07.376044 28909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:07.376149 28909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:07.376281 28909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:34:07.385190 28909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:07.386482 28909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:07.388242 28909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:07.400466 28914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:07.400451 28917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:07.400492 28915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:07.401862 28909 server_base.cc:1034] running on GCE node
I20260618 22:34:07.402437 28909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:07.403311 28909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:07.404510 28909 hybrid_clock.cc:648] HybridClock initialized: now 1781822047404462 us; error 44 us; skew 500 ppm
I20260618 22:34:07.404798 28909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:34:07.406888 28909 webserver.cc:460] Webserver started at http://127.18.150.129:46631/ using document root <none> and password file <none>
I20260618 22:34:07.407558 28909 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:07.407652 28909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:07.407948 28909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:07.410775 28909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "78a4f151804a43d986c881256373397c"
format_stamp: "Formatted at 2026-06-18 22:34:07 on dist-test-slave-h7jz"
server_key: "9db0df697f9b4af6fc1c14525c7e66b2"
server_key_iv: "37cf971dd58c5da8953f50335f5be6b2"
server_key_version: "encryptionkey@0"
I20260618 22:34:07.411497 28909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "78a4f151804a43d986c881256373397c"
format_stamp: "Formatted at 2026-06-18 22:34:07 on dist-test-slave-h7jz"
server_key: "9db0df697f9b4af6fc1c14525c7e66b2"
server_key_iv: "37cf971dd58c5da8953f50335f5be6b2"
server_key_version: "encryptionkey@0"
I20260618 22:34:07.416307 28909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:34:07.419566 28923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:07.420953 28909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:34:07.421105 28909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "78a4f151804a43d986c881256373397c"
format_stamp: "Formatted at 2026-06-18 22:34:07 on dist-test-slave-h7jz"
server_key: "9db0df697f9b4af6fc1c14525c7e66b2"
server_key_iv: "37cf971dd58c5da8953f50335f5be6b2"
server_key_version: "encryptionkey@0"
I20260618 22:34:07.421301 28909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:07.426703 28909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:07.427459 28909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:07.427700 28909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:07.428519 28909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:07.429785 28909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:07.429881 28909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:07.429996 28909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:07.430068 28909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:07.457580 28909 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:33857
I20260618 22:34:07.457616 29035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:33857 every 8 connection(s)
I20260618 22:34:07.458825 28909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:34:07.461436 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 28909
I20260618 22:34:07.461617 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:34:07.462383 19034 external_mini_cluster.cc:1428] Setting key b79af54355b160dcd6363e7876544c98
I20260618 22:34:07.465035 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39481
--builtin_ntp_servers=127.18.150.148:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260618 22:34:07.466943 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:07.461216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:37711 (local address 127.18.150.190:39481)
0618 22:34:07.461645 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:07.461650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:07.463116 (+ 1466us) server_negotiation.cc:407] Connection header received
0618 22:34:07.464864 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:07.464870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:07.464935 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:07.465057 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:07.466291 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:07.466296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:07.466300 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:07.466319 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:07.466371 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:07.466384 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:07.466487 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:07.466620 (+ 133us) server_negotiation.cc:299] Negotiation successful
0618 22:34:07.466677 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":199,"threads_started":1}
I20260618 22:34:07.467748 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:07.461989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39481 (local address 127.18.150.129:37711)
0618 22:34:07.462947 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:07.462992 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:07.464614 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:07.465259 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:07.465270 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:07.465383 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:07.466016 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:34:07.466019 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:07.466056 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:34:07.466143 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:07.466464 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:07.466470 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:07.466474 (+ 4us) client_negotiation.cc:769] Sending connection context
0618 22:34:07.466665 (+ 191us) client_negotiation.cc:240] Negotiation successful
0618 22:34:07.466854 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":297,"threads_started":1}
I20260618 22:34:07.469106 29036 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39481
I20260618 22:34:07.469403 29036 heartbeater.cc:464] Registering TS with master...
I20260618 22:34:07.470049 29036 heartbeater.cc:511] Master 127.18.150.190:39481 requested a full tablet report, sending...
I20260618 22:34:07.472164 28850 ts_manager.cc:194] Registered new tserver with Master: 78a4f151804a43d986c881256373397c (127.18.150.129:33857)
W20260618 22:34:07.779265 29040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:07.779532 29040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:07.779659 29040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:07.787098 29040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:07.787201 29040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:07.787350 29040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:34:07.796535 29040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:07.797802 29040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:07.799526 29040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:07.812114 29045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:07.812150 29048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:07.812184 29046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:07.813679 29040 server_base.cc:1034] running on GCE node
I20260618 22:34:07.814327 29040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:07.815264 29040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:07.816478 29040 hybrid_clock.cc:648] HybridClock initialized: now 1781822047816417 us; error 57 us; skew 500 ppm
I20260618 22:34:07.816813 29040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:34:07.819028 29040 webserver.cc:460] Webserver started at http://127.18.150.130:34249/ using document root <none> and password file <none>
I20260618 22:34:07.819705 29040 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:07.819804 29040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:07.820094 29040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:07.824860 29040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "c16dc5f2d3e04ea5bbfae43d08c8a645"
format_stamp: "Formatted at 2026-06-18 22:34:07 on dist-test-slave-h7jz"
server_key: "a6b5d02863b6387411f7962bb9ca7a5d"
server_key_iv: "1f3a399bf83650202c6869b14fa4535a"
server_key_version: "encryptionkey@0"
I20260618 22:34:07.825731 29040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "c16dc5f2d3e04ea5bbfae43d08c8a645"
format_stamp: "Formatted at 2026-06-18 22:34:07 on dist-test-slave-h7jz"
server_key: "a6b5d02863b6387411f7962bb9ca7a5d"
server_key_iv: "1f3a399bf83650202c6869b14fa4535a"
server_key_version: "encryptionkey@0"
I20260618 22:34:07.830924 29040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260618 22:34:07.834362 29054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:07.835815 29040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:34:07.835974 29040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "c16dc5f2d3e04ea5bbfae43d08c8a645"
format_stamp: "Formatted at 2026-06-18 22:34:07 on dist-test-slave-h7jz"
server_key: "a6b5d02863b6387411f7962bb9ca7a5d"
server_key_iv: "1f3a399bf83650202c6869b14fa4535a"
server_key_version: "encryptionkey@0"
I20260618 22:34:07.836175 29040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:07.841902 29040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:07.842736 29040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:07.842983 29040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:07.843887 29040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:07.845237 29040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:07.845332 29040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:07.845432 29040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:07.845497 29040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:07.875252 29040 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:42145
I20260618 22:34:07.875295 29166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:42145 every 8 connection(s)
I20260618 22:34:07.876636 29040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:34:07.878151 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 29040
I20260618 22:34:07.878341 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:34:07.878855 19034 external_mini_cluster.cc:1428] Setting key 8c9ffa02499c125e3bddbc0193e05077
I20260618 22:34:07.882016 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39481
--builtin_ntp_servers=127.18.150.148:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260618 22:34:07.886281 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:07.879477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:58509 (local address 127.18.150.190:39481)
0618 22:34:07.879685 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:07.879692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:07.881485 (+ 1793us) server_negotiation.cc:407] Connection header received
0618 22:34:07.883819 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:07.883824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:07.883881 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:07.883977 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:07.885458 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:07.885463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:07.885467 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:07.885484 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:07.885517 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:07.885528 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:07.885645 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:07.885911 (+ 266us) server_negotiation.cc:299] Negotiation successful
0618 22:34:07.885999 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260618 22:34:07.887233 29169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:07.880148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39481 (local address 127.18.150.130:58509)
0618 22:34:07.881311 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:07.881365 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:07.883498 (+ 2133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:07.884245 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:07.884257 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:07.884409 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:07.885093 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:34:07.885097 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:07.885145 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:34:07.885250 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:07.885651 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:07.885662 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:07.885671 (+ 9us) client_negotiation.cc:769] Sending connection context
0618 22:34:07.885935 (+ 264us) client_negotiation.cc:240] Negotiation successful
0618 22:34:07.886202 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":373,"threads_started":1}
I20260618 22:34:07.888592 29167 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39481
I20260618 22:34:07.888900 29167 heartbeater.cc:464] Registering TS with master...
I20260618 22:34:07.889554 29167 heartbeater.cc:511] Master 127.18.150.190:39481 requested a full tablet report, sending...
I20260618 22:34:07.890944 28850 ts_manager.cc:194] Registered new tserver with Master: c16dc5f2d3e04ea5bbfae43d08c8a645 (127.18.150.130:42145)
W20260618 22:34:08.199434 29170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:08.199718 29170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:08.199860 29170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:08.207113 29170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:08.207230 29170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:08.207371 29170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:34:08.216382 29170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:08.217603 29170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:08.219319 29170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:08.231940 29175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:08.231901 29178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:08.231972 29176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:08.232848 29170 server_base.cc:1034] running on GCE node
I20260618 22:34:08.233414 29170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:08.234314 29170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:08.235502 29170 hybrid_clock.cc:648] HybridClock initialized: now 1781822048235446 us; error 48 us; skew 500 ppm
I20260618 22:34:08.235796 29170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:34:08.237910 29170 webserver.cc:460] Webserver started at http://127.18.150.131:42203/ using document root <none> and password file <none>
I20260618 22:34:08.238597 29170 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:08.238677 29170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:08.238955 29170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:08.241837 29170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "6cf3a9119cd340c4bb75ebda93cbe4a6"
format_stamp: "Formatted at 2026-06-18 22:34:08 on dist-test-slave-h7jz"
server_key: "31d2c818c5efccca651c3f9ba299ca62"
server_key_iv: "7ae8d061e8880ae39349814001b1f18e"
server_key_version: "encryptionkey@0"
I20260618 22:34:08.242614 29170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "6cf3a9119cd340c4bb75ebda93cbe4a6"
format_stamp: "Formatted at 2026-06-18 22:34:08 on dist-test-slave-h7jz"
server_key: "31d2c818c5efccca651c3f9ba299ca62"
server_key_iv: "7ae8d061e8880ae39349814001b1f18e"
server_key_version: "encryptionkey@0"
I20260618 22:34:08.247529 29170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:34:08.250787 29184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:08.252065 29170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:34:08.252199 29170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "6cf3a9119cd340c4bb75ebda93cbe4a6"
format_stamp: "Formatted at 2026-06-18 22:34:08 on dist-test-slave-h7jz"
server_key: "31d2c818c5efccca651c3f9ba299ca62"
server_key_iv: "7ae8d061e8880ae39349814001b1f18e"
server_key_version: "encryptionkey@0"
I20260618 22:34:08.252400 29170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:08.257682 29170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:08.258435 29170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:08.258666 29170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:08.259480 29170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:08.260712 29170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:08.260802 29170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:08.260901 29170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:08.260968 29170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:08.289321 29170 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:40797
I20260618 22:34:08.289362 29296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:40797 every 8 connection(s)
I20260618 22:34:08.290558 29170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:34:08.295154 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 29170
I20260618 22:34:08.295312 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:34:08.295722 19034 external_mini_cluster.cc:1428] Setting key 1bf8e232efc5e6e04f3615b188b3e048
I20260618 22:34:08.298372 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:08.292932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:51157 (local address 127.18.150.190:39481)
0618 22:34:08.293112 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:08.293116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:08.294487 (+ 1371us) server_negotiation.cc:407] Connection header received
0618 22:34:08.296138 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:08.296143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:08.296197 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:08.296290 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:08.297661 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:08.297668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:08.297673 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:08.297691 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:08.297723 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:08.297734 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:08.297842 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:08.298028 (+ 186us) server_negotiation.cc:299] Negotiation successful
0618 22:34:08.298088 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:34:08.299357 29299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:08.293473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39481 (local address 127.18.150.131:51157)
0618 22:34:08.294352 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:08.294387 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:08.295886 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:08.296508 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:08.296516 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:08.296609 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:08.297334 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:34:08.297338 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:08.297388 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:34:08.297501 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:08.297817 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:08.297827 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:08.297835 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:34:08.298104 (+ 269us) client_negotiation.cc:240] Negotiation successful
0618 22:34:08.298355 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":246,"threads_started":1}
I20260618 22:34:08.300478 29297 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39481
I20260618 22:34:08.300715 29297 heartbeater.cc:464] Registering TS with master...
I20260618 22:34:08.301221 29297 heartbeater.cc:511] Master 127.18.150.190:39481 requested a full tablet report, sending...
I20260618 22:34:08.302399 28850 ts_manager.cc:194] Registered new tserver with Master: 6cf3a9119cd340c4bb75ebda93cbe4a6 (127.18.150.131:40797)
I20260618 22:34:08.310567 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260618 22:34:08.315862 29038 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:34:08.314525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.18.150.190:39481)
0618 22:34:08.314663 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:08.314668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:08.315139 (+ 471us) server_negotiation.cc:407] Connection header received
0618 22:34:08.315255 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:08.315258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:08.315306 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:08.315378 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:08.315613 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59754: BlockingRecv error: recv got EOF from 127.0.0.1:59754 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:34:08.316275 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 28909
I20260618 22:34:08.319761 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:08.317263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.18.150.129:33857)
0618 22:34:08.317742 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:08.317749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:08.317775 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:34:08.317827 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:08.317833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:08.318082 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:08.318219 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:08.319121 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:08.319127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:08.319138 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:08.319154 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:08.319188 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:08.319201 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:08.319321 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:08.319425 (+ 104us) server_negotiation.cc:299] Negotiation successful
0618 22:34:08.319517 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":218,"threads_started":1}
I20260618 22:34:08.320865 29031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:08.398636 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 28909
I20260618 22:34:08.410998 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 29040
I20260618 22:34:08.414218 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:08.411900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33058 (local address 127.18.150.130:42145)
0618 22:34:08.412349 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:08.412357 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:08.412382 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:34:08.412442 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:08.412448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:08.412688 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:08.412815 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:08.413593 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:08.413599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:08.413610 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:08.413621 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:08.413671 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:08.413688 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:08.413793 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:08.413885 (+ 92us) server_negotiation.cc:299] Negotiation successful
0618 22:34:08.413999 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":205,"threads_started":1}
I20260618 22:34:08.415355 29162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:08.494155 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 29040
I20260618 22:34:08.505900 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 29170
I20260618 22:34:08.509294 29310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:08.506739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51810 (local address 127.18.150.131:40797)
0618 22:34:08.507247 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:08.507254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:08.507283 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:34:08.507345 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:08.507351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:08.507561 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:08.507684 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:08.508591 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:08.508597 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:08.508609 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:34:08.508620 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:08.508660 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:34:08.508675 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:08.508796 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:08.508897 (+ 101us) server_negotiation.cc:299] Negotiation successful
0618 22:34:08.509000 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":234,"threads_started":1}
I20260618 22:34:08.510458 29292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:08.589257 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 29170
I20260618 22:34:08.603186 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 28819
I20260618 22:34:08.604307 28880 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:08.701519 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 28819
2026-06-18T22:34:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2198 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:34:08 dist-test-slave-h7jz krb5kdc[29315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:34:08 dist-test-slave-h7jz krb5kdc[29315](info): set up 2 sockets
Jun 18 22:34:08 dist-test-slave-h7jz krb5kdc[29315](info): commencing operation
krb5kdc: starting...
W20260618 22:34:09.714526 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.004s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:34:09 dist-test-slave-h7jz krb5kdc[29315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:34:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:34:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:34:09.923588 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38439
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:41141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:38439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:34:10.228365 29328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:10.228624 29328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:10.228708 29328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:10.235533 29328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:34:10.235622 29328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:10.235675 29328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:10.235726 29328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:34:10.235769 29328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:34:10.245257 29328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:41141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:38439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:10.246508 29328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:10.247713 29328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:10.258436 29334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:10.258585 29333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:10.258790 29336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:10.259225 29328 server_base.cc:1034] running on GCE node
I20260618 22:34:10.259905 29328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:10.261188 29328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:10.262382 29328 hybrid_clock.cc:648] HybridClock initialized: now 1781822050262299 us; error 72 us; skew 500 ppm
I20260618 22:34:10.262678 29328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:34:10 dist-test-slave-h7jz krb5kdc[29315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822050, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:34:10.265430 29328 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:34:10.266716 29328 webserver.cc:460] Webserver started at http://127.18.150.190:34739/ using document root <none> and password file <none>
I20260618 22:34:10.267323 29328 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:10.267432 29328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:10.267692 29328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:10.270462 29328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "b6bc185e50a84f2d8ae0b7d10c94f7dc"
format_stamp: "Formatted at 2026-06-18 22:34:10 on dist-test-slave-h7jz"
server_key: "446688bee0f056aafc1f08813951dda1"
server_key_iv: "c23992d58cb52c013fd6454cbdaa6008"
server_key_version: "encryptionkey@0"
I20260618 22:34:10.271137 29328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "b6bc185e50a84f2d8ae0b7d10c94f7dc"
format_stamp: "Formatted at 2026-06-18 22:34:10 on dist-test-slave-h7jz"
server_key: "446688bee0f056aafc1f08813951dda1"
server_key_iv: "c23992d58cb52c013fd6454cbdaa6008"
server_key_version: "encryptionkey@0"
I20260618 22:34:10.275695 29328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:34:10.278755 29343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:10.279935 29328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:34:10.280082 29328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "b6bc185e50a84f2d8ae0b7d10c94f7dc"
format_stamp: "Formatted at 2026-06-18 22:34:10 on dist-test-slave-h7jz"
server_key: "446688bee0f056aafc1f08813951dda1"
server_key_iv: "c23992d58cb52c013fd6454cbdaa6008"
server_key_version: "encryptionkey@0"
I20260618 22:34:10.280241 29328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:10.304821 29328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:10.308339 29328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:10.308583 29328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:10.324702 29328 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:38439
I20260618 22:34:10.324728 29394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:38439 every 8 connection(s)
I20260618 22:34:10.325930 29328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:34:10.329375 29395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:34:10.332420 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 29328
I20260618 22:34:10.332600 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:34:10.333231 19034 external_mini_cluster.cc:1428] Setting key 6e4ca294cada7c80d63522ab137bf78b
I20260618 22:34:10.340418 29395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc: Bootstrap starting.
I20260618 22:34:10.343235 29395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc: Neither blocks nor log segments found. Creating new log.
I20260618 22:34:10.344406 29395 log.cc:824] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:34:10 dist-test-slave-h7jz krb5kdc[29315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:34:10.347311 29395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc: No bootstrap required, opened a new log
I20260618 22:34:10.354825 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:10.335096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35610 (local address 127.18.150.190:38439)
0618 22:34:10.335807 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:10.335819 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:10.335859 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:34:10.337330 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:10.337352 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:10.337735 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:10.338123 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:10.339458 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:10.340731 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:10.342029 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:10.342658 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:10.342825 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:10.346969 (+ 4144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:10.346980 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:10.346997 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:10.347036 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:10.350698 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:10.351302 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:10.351310 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:10.351317 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:10.351520 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:10.351823 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:10.351828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:10.351832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:10.352483 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:10.352790 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:10.353045 (+ 255us) server_negotiation.cc:299] Negotiation successful
0618 22:34:10.353478 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":305,"threads_started":1}
I20260618 22:34:10.357228 29395 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6bc185e50a84f2d8ae0b7d10c94f7dc" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38439 } }
I20260618 22:34:10.357611 29395 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:34:10.357709 29395 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6bc185e50a84f2d8ae0b7d10c94f7dc, State: Initialized, Role: FOLLOWER
I20260618 22:34:10.358242 29395 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6bc185e50a84f2d8ae0b7d10c94f7dc" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38439 } }
I20260618 22:34:10.358443 29395 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:34:10.358559 29395 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:34:10.358698 29395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:34:10.360841 29395 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6bc185e50a84f2d8ae0b7d10c94f7dc" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38439 } }
I20260618 22:34:10.361270 29395 leader_election.cc:302] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6bc185e50a84f2d8ae0b7d10c94f7dc; no voters:
I20260618 22:34:10.361785 29395 leader_election.cc:288] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:34:10.361891 29400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:34:10.362500 29400 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [term 1 LEADER]: Becoming Leader. State: Replica: b6bc185e50a84f2d8ae0b7d10c94f7dc, State: Running, Role: LEADER
I20260618 22:34:10.362977 29400 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6bc185e50a84f2d8ae0b7d10c94f7dc" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38439 } }
I20260618 22:34:10.363577 29395 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:34:10.365563 29401 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6bc185e50a84f2d8ae0b7d10c94f7dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6bc185e50a84f2d8ae0b7d10c94f7dc" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38439 } } }
I20260618 22:34:10.365921 29401 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [sys.catalog]: This master's current role is: LEADER
I20260618 22:34:10.367709 29402 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6bc185e50a84f2d8ae0b7d10c94f7dc. Latest consensus state: current_term: 1 leader_uuid: "b6bc185e50a84f2d8ae0b7d10c94f7dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6bc185e50a84f2d8ae0b7d10c94f7dc" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38439 } } }
I20260618 22:34:10.368036 29402 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc [sys.catalog]: This master's current role is: LEADER
I20260618 22:34:10.368180 29409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:34:10.372112 29409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:34:10.379666 29409 catalog_manager.cc:1347] Generated new cluster ID: c5ef8c0d97e4431da7632477f1559b08
I20260618 22:34:10.379772 29409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:34:10.411316 29409 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:34:10.412606 29409 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:34:10.418540 29409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc: Generated new TSK 0
I20260618 22:34:10.419297 29409 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:34:10.481756 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38439
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:41141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:34:10.790772 29421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:10.791049 29421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:10.791170 29421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:10.798345 29421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:10.798457 29421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:10.798588 29421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:34:10.808169 29421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:41141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:10.809441 29421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:10.810750 29421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:10.824394 29426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:10.824429 29429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:10.824472 29427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:10.824965 29421 server_base.cc:1034] running on GCE node
I20260618 22:34:10.825542 29421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:10.826531 29421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:10.827732 29421 hybrid_clock.cc:648] HybridClock initialized: now 1781822050827683 us; error 78 us; skew 500 ppm
I20260618 22:34:10.828049 29421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:34:10 dist-test-slave-h7jz krb5kdc[29315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822050, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:34:10.830928 29421 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:34:10.832343 29421 webserver.cc:460] Webserver started at http://127.18.150.129:41701/ using document root <none> and password file <none>
I20260618 22:34:10.832980 29421 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:10.833076 29421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:10.833323 29421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:10.836161 29421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "b2ca9d5d1639417a9f41e48c973fc09a"
format_stamp: "Formatted at 2026-06-18 22:34:10 on dist-test-slave-h7jz"
server_key: "bc222b59e0cc81fc1b9f76671f58da7c"
server_key_iv: "c4747ab012ebbb033f7a9c280bc27283"
server_key_version: "encryptionkey@0"
I20260618 22:34:10.836881 29421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "b2ca9d5d1639417a9f41e48c973fc09a"
format_stamp: "Formatted at 2026-06-18 22:34:10 on dist-test-slave-h7jz"
server_key: "bc222b59e0cc81fc1b9f76671f58da7c"
server_key_iv: "c4747ab012ebbb033f7a9c280bc27283"
server_key_version: "encryptionkey@0"
I20260618 22:34:10.841683 29421 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:34:10.844887 29436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:10.846194 29421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:34:10.846328 29421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "b2ca9d5d1639417a9f41e48c973fc09a"
format_stamp: "Formatted at 2026-06-18 22:34:10 on dist-test-slave-h7jz"
server_key: "bc222b59e0cc81fc1b9f76671f58da7c"
server_key_iv: "c4747ab012ebbb033f7a9c280bc27283"
server_key_version: "encryptionkey@0"
I20260618 22:34:10.846514 29421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:10.878060 29421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:10.882275 29421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:10.882545 29421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:10.883558 29421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:10.884965 29421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:10.885035 29421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:10.885128 29421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:10.885169 29421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:10.913636 29421 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:46761
I20260618 22:34:10.913674 29548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:46761 every 8 connection(s)
I20260618 22:34:10.914883 29421 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:34:10.924702 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 29421
I20260618 22:34:10.924916 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:34:10.925591 19034 external_mini_cluster.cc:1428] Setting key 96080173cae6abd631b55c4d3572f056
Jun 18 22:34:10 dist-test-slave-h7jz krb5kdc[29315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822050, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:34:10.934388 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:10.917212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:40527 (local address 127.18.150.190:38439)
0618 22:34:10.917771 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:10.917778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:10.918723 (+ 945us) server_negotiation.cc:407] Connection header received
0618 22:34:10.920577 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:10.920582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:10.920651 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:10.920739 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:10.922184 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:10.922885 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:10.924385 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:10.924963 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:10.925088 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:10.928845 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:10.928850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:10.928853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:10.928885 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:10.931259 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:10.932073 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:10.932078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:10.932080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:10.932199 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:10.932634 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:10.932639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:10.932642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:10.933025 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:10.933174 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:10.933928 (+ 754us) server_negotiation.cc:299] Negotiation successful
0618 22:34:10.934133 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":259,"threads_started":1}
I20260618 22:34:10.935462 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:10.917810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38439 (local address 127.18.150.129:40527)
0618 22:34:10.918563 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:10.918606 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:10.920215 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:10.920954 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:10.920962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:10.921442 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:10.922026 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:10.922045 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:10.923025 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:10.923029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:10.924241 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:10.924252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:10.925159 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:10.925164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:10.925359 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:10.926041 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:10.926065 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:10.928639 (+ 2574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:10.931396 (+ 2757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:10.931405 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:10.931419 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:10.931933 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:10.932293 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:10.932297 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:10.932301 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:10.932517 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:10.933142 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:10.933150 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:10.933538 (+ 388us) client_negotiation.cc:769] Sending connection context
0618 22:34:10.933812 (+ 274us) client_negotiation.cc:240] Negotiation successful
0618 22:34:10.934352 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":351,"threads_started":1}
I20260618 22:34:10.936935 29549 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38439
I20260618 22:34:10.937235 29549 heartbeater.cc:464] Registering TS with master...
I20260618 22:34:10.938055 29549 heartbeater.cc:511] Master 127.18.150.190:38439 requested a full tablet report, sending...
I20260618 22:34:10.940276 29360 ts_manager.cc:194] Registered new tserver with Master: b2ca9d5d1639417a9f41e48c973fc09a (127.18.150.129:46761)
I20260618 22:34:10.942121 29360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:40527
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:34:10.974906 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38439
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:41141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:34:11.278486 29555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:11.278746 29555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:11.278872 29555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:11.285754 29555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:11.285845 29555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:11.286003 29555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:34:11.295089 29555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:41141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:11.296291 29555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:11.297528 29555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:11.309370 29561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:11.309405 29560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:11.309425 29563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:11.310273 29555 server_base.cc:1034] running on GCE node
I20260618 22:34:11.310787 29555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:11.311568 29555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:11.312772 29555 hybrid_clock.cc:648] HybridClock initialized: now 1781822051312721 us; error 50 us; skew 500 ppm
I20260618 22:34:11.313022 29555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:34:11 dist-test-slave-h7jz krb5kdc[29315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822051, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:34:11.320369 29555 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:34:11.321719 29555 webserver.cc:460] Webserver started at http://127.18.150.130:38295/ using document root <none> and password file <none>
I20260618 22:34:11.322402 29555 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:11.322505 29555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:11.322774 29555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:11.325582 29555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "049663b848c14b7b81ab01448100c7ab"
format_stamp: "Formatted at 2026-06-18 22:34:11 on dist-test-slave-h7jz"
server_key: "939609d7f89061ca7b6e923b29523da0"
server_key_iv: "0b21e0260423ac1ccdc3f061cca0ba33"
server_key_version: "encryptionkey@0"
I20260618 22:34:11.326320 29555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "049663b848c14b7b81ab01448100c7ab"
format_stamp: "Formatted at 2026-06-18 22:34:11 on dist-test-slave-h7jz"
server_key: "939609d7f89061ca7b6e923b29523da0"
server_key_iv: "0b21e0260423ac1ccdc3f061cca0ba33"
server_key_version: "encryptionkey@0"
I20260618 22:34:11.331069 29555 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260618 22:34:11.334297 29570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:11.335711 29555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:34:11.335856 29555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "049663b848c14b7b81ab01448100c7ab"
format_stamp: "Formatted at 2026-06-18 22:34:11 on dist-test-slave-h7jz"
server_key: "939609d7f89061ca7b6e923b29523da0"
server_key_iv: "0b21e0260423ac1ccdc3f061cca0ba33"
server_key_version: "encryptionkey@0"
I20260618 22:34:11.336024 29555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:11.357069 29555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:11.360610 29555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:11.360859 29555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:11.361696 29555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:11.362960 29555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:11.363035 29555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:11.363147 29555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:11.363197 29555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:11.391317 29555 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:43319
I20260618 22:34:11.391364 29682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:43319 every 8 connection(s)
I20260618 22:34:11.392482 29555 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:34:11.393921 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 29555
I20260618 22:34:11.394124 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:34:11.394635 19034 external_mini_cluster.cc:1428] Setting key b9bc23fdd2ba4be05144b8110378178a
Jun 18 22:34:11 dist-test-slave-h7jz krb5kdc[29315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822051, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:34:11.412796 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:11.395060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:52765 (local address 127.18.150.190:38439)
0618 22:34:11.395228 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:11.395233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:11.396515 (+ 1282us) server_negotiation.cc:407] Connection header received
0618 22:34:11.398930 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:11.398934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:11.398993 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:11.399065 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:11.400974 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.401858 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.403861 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.404250 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.404338 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:11.407605 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:11.407610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:11.407613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:11.407649 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:11.410032 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:11.410698 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:11.410702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:11.410705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:11.410828 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:11.411216 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:11.411220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:11.411222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:11.411472 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:11.411593 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:11.412250 (+ 657us) server_negotiation.cc:299] Negotiation successful
0618 22:34:11.412461 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:34:11.413789 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:11.395604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38439 (local address 127.18.150.130:52765)
0618 22:34:11.396385 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:11.396428 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:11.398622 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:11.399318 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:11.399329 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:11.399995 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:11.400800 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:11.400824 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.402026 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.402031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:11.403739 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:11.403753 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.404485 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.404492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:11.404772 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:11.405388 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:11.405413 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:11.407394 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:11.410162 (+ 2768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:11.410171 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:11.410188 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:11.410577 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:11.410923 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:11.410928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:11.410933 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:11.411105 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:11.411586 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:11.411594 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:11.411913 (+ 319us) client_negotiation.cc:769] Sending connection context
0618 22:34:11.412135 (+ 222us) client_negotiation.cc:240] Negotiation successful
0618 22:34:11.412655 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":315,"threads_started":1}
I20260618 22:34:11.415180 29683 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38439
I20260618 22:34:11.415462 29683 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:34:11.416139 29683 heartbeater.cc:511] Master 127.18.150.190:38439 requested a full tablet report, sending...
I20260618 22:34:11.417495 29360 ts_manager.cc:194] Registered new tserver with Master: 049663b848c14b7b81ab01448100c7ab (127.18.150.130:43319)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
I20260618 22:34:11.418360 29360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:52765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:34:11.442268 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38439
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:41141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:34:11.749275 29688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:11.749536 29688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:11.749631 29688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:11.756896 29688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:11.757010 29688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:11.757179 29688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:34:11.766860 29688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:41141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:11.768123 29688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:11.769428 29688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:11.782262 29696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:11.782297 29694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:11.782322 29693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:11.782804 29688 server_base.cc:1034] running on GCE node
I20260618 22:34:11.783449 29688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:11.784387 29688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:11.785571 29688 hybrid_clock.cc:648] HybridClock initialized: now 1781822051785524 us; error 38 us; skew 500 ppm
I20260618 22:34:11.786036 29688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:34:11 dist-test-slave-h7jz krb5kdc[29315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822051, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:34:11.789019 29688 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:34:11.790499 29688 webserver.cc:460] Webserver started at http://127.18.150.131:34083/ using document root <none> and password file <none>
I20260618 22:34:11.791160 29688 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:11.791273 29688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:11.791550 29688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:34:11.794431 29688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "cd649195a8c2481face8aa07a294cd0e"
format_stamp: "Formatted at 2026-06-18 22:34:11 on dist-test-slave-h7jz"
server_key: "a75199f9a19f020898e7bbe2ac0e74b0"
server_key_iv: "597ffa6a257eb23bbf5d34b35f53d30e"
server_key_version: "encryptionkey@0"
I20260618 22:34:11.795178 29688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "cd649195a8c2481face8aa07a294cd0e"
format_stamp: "Formatted at 2026-06-18 22:34:11 on dist-test-slave-h7jz"
server_key: "a75199f9a19f020898e7bbe2ac0e74b0"
server_key_iv: "597ffa6a257eb23bbf5d34b35f53d30e"
server_key_version: "encryptionkey@0"
I20260618 22:34:11.800300 29688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260618 22:34:11.803750 29703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:11.805089 29688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:34:11.805236 29688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "cd649195a8c2481face8aa07a294cd0e"
format_stamp: "Formatted at 2026-06-18 22:34:11 on dist-test-slave-h7jz"
server_key: "a75199f9a19f020898e7bbe2ac0e74b0"
server_key_iv: "597ffa6a257eb23bbf5d34b35f53d30e"
server_key_version: "encryptionkey@0"
I20260618 22:34:11.805405 29688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:11.827353 29688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:11.831311 29688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:11.831580 29688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:11.832507 29688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:11.833829 29688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:11.833909 29688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:11.834044 29688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:11.834096 29688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:11.864037 29688 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:33295
I20260618 22:34:11.864102 29815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:33295 every 8 connection(s)
I20260618 22:34:11.865387 29688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:34:11.871484 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 29688
I20260618 22:34:11.871692 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:34:11.872222 19034 external_mini_cluster.cc:1428] Setting key 8d7bb3d38bb52822b2cd91c886245e9a
Jun 18 22:34:11 dist-test-slave-h7jz krb5kdc[29315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822051, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:34:11.884776 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:11.867981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:36517 (local address 127.18.150.190:38439)
0618 22:34:11.868167 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:11.868174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:11.869387 (+ 1213us) server_negotiation.cc:407] Connection header received
0618 22:34:11.871155 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:11.871159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:11.871227 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:11.871312 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:11.873417 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.874194 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.875679 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.876062 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.876165 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:11.879253 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:11.879258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:11.879261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:11.879304 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:11.881811 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:11.882527 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:11.882532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:11.882535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:11.882648 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:11.883070 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:11.883074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:11.883076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:11.883383 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:11.883509 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:11.884205 (+ 696us) server_negotiation.cc:299] Negotiation successful
0618 22:34:11.884440 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:34:11.885727 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:11.868531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38439 (local address 127.18.150.131:36517)
0618 22:34:11.869220 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:11.869274 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:11.870898 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:11.871586 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:11.871600 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:11.872325 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:11.873081 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:11.873113 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.874333 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.874337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:11.875547 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:11.875559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.876260 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.876267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:11.876483 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:11.877044 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:11.877066 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:11.879053 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:11.881972 (+ 2919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:11.881982 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:11.882000 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:11.882416 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:11.882745 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:11.882750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:11.882754 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:11.882963 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:11.883502 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:11.883511 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:11.883861 (+ 350us) client_negotiation.cc:769] Sending connection context
0618 22:34:11.884107 (+ 246us) client_negotiation.cc:240] Negotiation successful
0618 22:34:11.884645 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":266,"threads_started":1}
I20260618 22:34:11.887060 29816 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38439
I20260618 22:34:11.887343 29816 heartbeater.cc:464] Registering TS with master...
I20260618 22:34:11.888024 29816 heartbeater.cc:511] Master 127.18.150.190:38439 requested a full tablet report, sending...
I20260618 22:34:11.889353 29360 ts_manager.cc:194] Registered new tserver with Master: cd649195a8c2481face8aa07a294cd0e (127.18.150.131:33295)
I20260618 22:34:11.890211 29360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:36517
I20260618 22:34:11.899580 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:34:11.916098 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:11.903922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35624 (local address 127.18.150.190:38439)
0618 22:34:11.904095 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:11.904103 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:11.904521 (+ 418us) server_negotiation.cc:407] Connection header received
0618 22:34:11.904862 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:11.904875 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:11.904943 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:11.905025 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:11.906486 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.907189 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.908417 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.908841 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.908945 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:11.911110 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:11.911115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:11.911118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:11.911152 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:11.913518 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:11.914163 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:11.914167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:11.914170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:11.914295 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:11.914686 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:11.914692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:11.914695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:11.915087 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:11.915322 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:11.915572 (+ 250us) server_negotiation.cc:299] Negotiation successful
0618 22:34:11.915780 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:34:11.920058 29360 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:34:11.926788 29360 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:34:11.945452 29549 heartbeater.cc:503] Master 127.18.150.190:38439 was elected leader, sending a full tablet report...
I20260618 22:34:11.950048 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:11.937772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53324 (local address 127.18.150.131:33295)
0618 22:34:11.942494 (+ 4722us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:11.942503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:11.942537 (+ 34us) server_negotiation.cc:407] Connection header received
0618 22:34:11.942639 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:11.942647 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:11.942903 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:11.943820 (+ 917us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:34:11.944086 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.945448 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.948040 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.948958 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.949129 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:11.949302 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:11.949552 (+ 250us) server_negotiation.cc:299] Negotiation successful
0618 22:34:11.949704 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4544,"thread_start_us":4355,"threads_started":1}
I20260618 22:34:11.950044 29828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:11.937452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:33295 (local address 127.0.0.1:53324)
0618 22:34:11.938318 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:11.938357 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:11.938513 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:11.943143 (+ 4630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:11.943153 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:11.943185 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:34:11.943636 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:11.943648 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.945587 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.945592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:11.947886 (+ 2294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:11.947901 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.949144 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.949150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:11.949351 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:11.949399 (+ 48us) client_negotiation.cc:769] Sending connection context
0618 22:34:11.949507 (+ 108us) client_negotiation.cc:240] Negotiation successful
0618 22:34:11.949690 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":517,"threads_started":1}
I20260618 22:34:11.951917 29831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:11.938781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:43319 (local address 127.0.0.1:51924)
0618 22:34:11.943396 (+ 4615us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:11.943411 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:11.943512 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:11.944171 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:11.944175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:11.944199 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:34:11.944742 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:11.944761 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.946731 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.946739 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:11.948957 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:11.948971 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.951086 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.951092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:11.951240 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:11.951347 (+ 107us) client_negotiation.cc:769] Sending connection context
0618 22:34:11.951425 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:34:11.951542 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4475,"mutex_wait_us":112,"thread_start_us":4070,"threads_started":1}
I20260618 22:34:11.952396 29832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:11.938041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:46761 (local address 127.0.0.1:42654)
0618 22:34:11.943091 (+ 5050us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:11.943110 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:11.943314 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:11.944160 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:11.944164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:11.944186 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:34:11.944690 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:11.944699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.946617 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.946622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:11.949606 (+ 2984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:11.949619 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.951648 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.951654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:11.951807 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:11.951837 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:34:11.951936 (+ 99us) client_negotiation.cc:240] Negotiation successful
0618 22:34:11.952048 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4929,"mutex_wait_us":1419,"thread_start_us":4466,"threads_started":1}
I20260618 22:34:11.952412 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:11.938451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.18.150.129:46761)
0618 22:34:11.939288 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:11.939298 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:11.943209 (+ 3911us) server_negotiation.cc:407] Connection header received
0618 22:34:11.943518 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:11.943526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:11.943735 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:11.943908 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:34:11.944848 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.946479 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.949763 (+ 3284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.951175 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.951337 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:11.951480 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:11.951979 (+ 499us) server_negotiation.cc:299] Negotiation successful
0618 22:34:11.952128 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":572,"threads_started":1}
I20260618 22:34:11.952653 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:11.939127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51924 (local address 127.18.150.130:43319)
0618 22:34:11.942700 (+ 3573us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:11.942710 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:11.943707 (+ 997us) server_negotiation.cc:407] Connection header received
0618 22:34:11.943797 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:11.943803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:11.944001 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:11.944168 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:34:11.944918 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.946462 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.949126 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:11.950627 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:11.950787 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:11.950943 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:11.952206 (+ 1263us) server_negotiation.cc:299] Negotiation successful
0618 22:34:11.952364 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3420,"thread_start_us":3296,"threads_started":1}
I20260618 22:34:11.954372 29751 tablet_service.cc:1467] Processing CreateTablet for tablet 8298c3ff2b684d23b3ecf34ebf440b4a (DEFAULT_TABLE table=test-table [id=f4716aa270e34f209e3b658168b0a6fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:34:11.955796 29751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8298c3ff2b684d23b3ecf34ebf440b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:34:11.955547 29484 tablet_service.cc:1467] Processing CreateTablet for tablet 8298c3ff2b684d23b3ecf34ebf440b4a (DEFAULT_TABLE table=test-table [id=f4716aa270e34f209e3b658168b0a6fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:34:11.955760 29618 tablet_service.cc:1467] Processing CreateTablet for tablet 8298c3ff2b684d23b3ecf34ebf440b4a (DEFAULT_TABLE table=test-table [id=f4716aa270e34f209e3b658168b0a6fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:34:11.956596 29484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8298c3ff2b684d23b3ecf34ebf440b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:34:11.956961 29618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8298c3ff2b684d23b3ecf34ebf440b4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:34:11.969439 29835 tablet_bootstrap.cc:492] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: Bootstrap starting.
I20260618 22:34:11.973233 29835 tablet_bootstrap.cc:654] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: Neither blocks nor log segments found. Creating new log.
I20260618 22:34:11.973973 29834 tablet_bootstrap.cc:492] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: Bootstrap starting.
I20260618 22:34:11.974128 29836 tablet_bootstrap.cc:492] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: Bootstrap starting.
I20260618 22:34:11.975037 29835 log.cc:824] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:34:11.977073 29834 tablet_bootstrap.cc:654] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: Neither blocks nor log segments found. Creating new log.
I20260618 22:34:11.977092 29836 tablet_bootstrap.cc:654] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: Neither blocks nor log segments found. Creating new log.
I20260618 22:34:11.978092 29836 log.cc:824] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: Log is configured to *not* fsync() on all Append() calls
I20260618 22:34:11.978240 29834 log.cc:824] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:34:11.979031 29835 tablet_bootstrap.cc:492] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: No bootstrap required, opened a new log
I20260618 22:34:11.979298 29835 ts_tablet_manager.cc:1397] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260618 22:34:11.980701 29836 tablet_bootstrap.cc:492] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: No bootstrap required, opened a new log
I20260618 22:34:11.980965 29836 ts_tablet_manager.cc:1397] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260618 22:34:11.981426 29834 tablet_bootstrap.cc:492] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: No bootstrap required, opened a new log
I20260618 22:34:11.981647 29834 ts_tablet_manager.cc:1397] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260618 22:34:11.986738 29835 raft_consensus.cc:348] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd649195a8c2481face8aa07a294cd0e" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33295 } } peers { permanent_uuid: "049663b848c14b7b81ab01448100c7ab" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43319 } } peers { permanent_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 46761 } }
I20260618 22:34:11.987062 29835 raft_consensus.cc:374] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:34:11.987145 29835 raft_consensus.cc:729] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd649195a8c2481face8aa07a294cd0e, State: Initialized, Role: FOLLOWER
I20260618 22:34:11.987777 29835 consensus_queue.cc:260] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd649195a8c2481face8aa07a294cd0e" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33295 } } peers { permanent_uuid: "049663b848c14b7b81ab01448100c7ab" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43319 } } peers { permanent_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 46761 } }
I20260618 22:34:11.988730 29816 heartbeater.cc:503] Master 127.18.150.190:38439 was elected leader, sending a full tablet report...
I20260618 22:34:11.989372 29835 ts_tablet_manager.cc:1428] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260618 22:34:11.990923 29836 raft_consensus.cc:348] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd649195a8c2481face8aa07a294cd0e" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33295 } } peers { permanent_uuid: "049663b848c14b7b81ab01448100c7ab" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43319 } } peers { permanent_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 46761 } }
I20260618 22:34:11.991321 29836 raft_consensus.cc:374] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:34:11.991426 29836 raft_consensus.cc:729] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049663b848c14b7b81ab01448100c7ab, State: Initialized, Role: FOLLOWER
I20260618 22:34:11.991540 29834 raft_consensus.cc:348] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd649195a8c2481face8aa07a294cd0e" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33295 } } peers { permanent_uuid: "049663b848c14b7b81ab01448100c7ab" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43319 } } peers { permanent_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 46761 } }
I20260618 22:34:11.991914 29834 raft_consensus.cc:374] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:34:11.992004 29834 raft_consensus.cc:729] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2ca9d5d1639417a9f41e48c973fc09a, State: Initialized, Role: FOLLOWER
I20260618 22:34:11.991995 29836 consensus_queue.cc:260] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd649195a8c2481face8aa07a294cd0e" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33295 } } peers { permanent_uuid: "049663b848c14b7b81ab01448100c7ab" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43319 } } peers { permanent_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 46761 } }
I20260618 22:34:11.992615 29834 consensus_queue.cc:260] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd649195a8c2481face8aa07a294cd0e" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33295 } } peers { permanent_uuid: "049663b848c14b7b81ab01448100c7ab" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43319 } } peers { permanent_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 46761 } }
I20260618 22:34:11.993017 29683 heartbeater.cc:503] Master 127.18.150.190:38439 was elected leader, sending a full tablet report...
I20260618 22:34:11.993604 29836 ts_tablet_manager.cc:1428] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
I20260618 22:34:11.994165 29834 ts_tablet_manager.cc:1428] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: Time spent starting tablet: real 0.012s user 0.012s sys 0.002s
I20260618 22:34:12.074345 29840 raft_consensus.cc:482] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:34:12.074609 29840 raft_consensus.cc:504] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd649195a8c2481face8aa07a294cd0e" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33295 } } peers { permanent_uuid: "049663b848c14b7b81ab01448100c7ab" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43319 } } peers { permanent_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 46761 } }
I20260618 22:34:12.076239 29840 leader_election.cc:288] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 049663b848c14b7b81ab01448100c7ab (127.18.150.130:43319), b2ca9d5d1639417a9f41e48c973fc09a (127.18.150.129:46761)
I20260618 22:34:12.082016 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:12.076625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:49787 (local address 127.18.150.130:43319)
0618 22:34:12.076932 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:12.076937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:12.076957 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:34:12.077026 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:12.077032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:12.077092 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:12.077169 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:34:12.077566 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.078524 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.080214 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.081189 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.081298 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:12.081349 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:12.081655 (+ 306us) server_negotiation.cc:299] Negotiation successful
0618 22:34:12.081736 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260618 22:34:12.082067 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:12.076427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:43319 (local address 127.18.150.131:49787)
0618 22:34:12.076613 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:12.076631 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:12.076750 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:12.077212 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:12.077217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:12.077243 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:34:12.077416 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:12.077427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.078643 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.078649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:12.080065 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:12.080076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.081321 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.081334 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:12.081514 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:12.081549 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:34:12.081642 (+ 93us) client_negotiation.cc:240] Negotiation successful
0618 22:34:12.081766 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260618 22:34:12.082641 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:12.076809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:55049 (local address 127.18.150.129:46761)
0618 22:34:12.076988 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:12.076993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:12.077411 (+ 418us) server_negotiation.cc:407] Connection header received
0618 22:34:12.077543 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:12.077549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:12.077610 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:12.077673 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:34:12.078216 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.079053 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.080838 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.081732 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.081872 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:12.081956 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:12.082303 (+ 347us) server_negotiation.cc:299] Negotiation successful
0618 22:34:12.082423 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:34:12.082683 29843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:12.076644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:46761 (local address 127.18.150.131:55049)
0618 22:34:12.077314 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:12.077332 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:12.077436 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:12.077700 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:12.077705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:12.077727 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:34:12.078087 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:12.078099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.079184 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.079188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:12.080682 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:12.080695 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.081911 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.081917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:12.082106 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:12.082138 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:34:12.082271 (+ 133us) client_negotiation.cc:240] Negotiation successful
0618 22:34:12.082389 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":359,"threads_started":1}
I20260618 22:34:12.083079 29638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8298c3ff2b684d23b3ecf34ebf440b4a" candidate_uuid: "cd649195a8c2481face8aa07a294cd0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "049663b848c14b7b81ab01448100c7ab" is_pre_election: true
I20260618 22:34:12.083519 29638 raft_consensus.cc:2454] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd649195a8c2481face8aa07a294cd0e in term 0.
I20260618 22:34:12.083515 29504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8298c3ff2b684d23b3ecf34ebf440b4a" candidate_uuid: "cd649195a8c2481face8aa07a294cd0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" is_pre_election: true
I20260618 22:34:12.084031 29504 raft_consensus.cc:2454] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd649195a8c2481face8aa07a294cd0e in term 0.
I20260618 22:34:12.084164 29704 leader_election.cc:302] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 049663b848c14b7b81ab01448100c7ab, cd649195a8c2481face8aa07a294cd0e; no voters:
I20260618 22:34:12.084590 29840 raft_consensus.cc:2781] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:34:12.084707 29840 raft_consensus.cc:482] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:34:12.084812 29840 raft_consensus.cc:3037] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:34:12.086951 29840 raft_consensus.cc:504] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd649195a8c2481face8aa07a294cd0e" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33295 } } peers { permanent_uuid: "049663b848c14b7b81ab01448100c7ab" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43319 } } peers { permanent_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 46761 } }
I20260618 22:34:12.087482 29840 leader_election.cc:288] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [CANDIDATE]: Term 1 election: Requested vote from peers 049663b848c14b7b81ab01448100c7ab (127.18.150.130:43319), b2ca9d5d1639417a9f41e48c973fc09a (127.18.150.129:46761)
I20260618 22:34:12.087865 29638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8298c3ff2b684d23b3ecf34ebf440b4a" candidate_uuid: "cd649195a8c2481face8aa07a294cd0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "049663b848c14b7b81ab01448100c7ab"
I20260618 22:34:12.087873 29504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8298c3ff2b684d23b3ecf34ebf440b4a" candidate_uuid: "cd649195a8c2481face8aa07a294cd0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2ca9d5d1639417a9f41e48c973fc09a"
I20260618 22:34:12.088017 29504 raft_consensus.cc:3037] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:34:12.088027 29638 raft_consensus.cc:3037] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:34:12.090040 29638 raft_consensus.cc:2454] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd649195a8c2481face8aa07a294cd0e in term 1.
I20260618 22:34:12.090042 29504 raft_consensus.cc:2454] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd649195a8c2481face8aa07a294cd0e in term 1.
I20260618 22:34:12.090368 29706 leader_election.cc:302] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2ca9d5d1639417a9f41e48c973fc09a, cd649195a8c2481face8aa07a294cd0e; no voters:
I20260618 22:34:12.090644 29840 raft_consensus.cc:2781] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:34:12.091025 29840 raft_consensus.cc:686] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 1 LEADER]: Becoming Leader. State: Replica: cd649195a8c2481face8aa07a294cd0e, State: Running, Role: LEADER
I20260618 22:34:12.091506 29840 consensus_queue.cc:237] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd649195a8c2481face8aa07a294cd0e" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33295 } } peers { permanent_uuid: "049663b848c14b7b81ab01448100c7ab" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43319 } } peers { permanent_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 46761 } }
I20260618 22:34:12.095656 29358 catalog_manager.cc:5495] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e reported cstate change: term changed from 0 to 1, leader changed from <none> to cd649195a8c2481face8aa07a294cd0e (127.18.150.131). New cstate: current_term: 1 leader_uuid: "cd649195a8c2481face8aa07a294cd0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd649195a8c2481face8aa07a294cd0e" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "049663b848c14b7b81ab01448100c7ab" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 46761 } health_report { overall_health: UNKNOWN } } }
W20260618 22:34:12.116909 29817 tablet.cc:2369] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:34:12.144227 29684 tablet.cc:2369] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:34:12.145565 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:12.140890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53326 (local address 127.18.150.131:33295)
0618 22:34:12.141063 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:12.141068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:12.141083 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:34:12.141161 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:12.141165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:12.141242 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:12.141366 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:34:12.141637 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.142370 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.143610 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.143993 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.144105 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:12.144536 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:34:12.145027 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:34:12.145156 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:12.145256 (+ 100us) server_negotiation.cc:299] Negotiation successful
0618 22:34:12.145333 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:34:12.153257 29638 raft_consensus.cc:1261] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab [term 1 FOLLOWER]: Refusing update from remote peer cd649195a8c2481face8aa07a294cd0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:34:12.153266 29504 raft_consensus.cc:1261] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a [term 1 FOLLOWER]: Refusing update from remote peer cd649195a8c2481face8aa07a294cd0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:34:12.154141 29844 consensus_queue.cc:1035] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2ca9d5d1639417a9f41e48c973fc09a" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 46761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:34:12.154461 29840 consensus_queue.cc:1035] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "049663b848c14b7b81ab01448100c7ab" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:34:12.167097 29550 tablet.cc:2369] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:34:12.171818 29849 mvcc.cc:205] Tried to move back new op lower bound from 7298343125610369024 to 7298343125370425344. Current Snapshot: MvccSnapshot[applied={T|T < 7298343125610369024}]
I20260618 22:34:12.177068 29360 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:34:12.177322 29360 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:34:12.180466 29360 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6bc185e50a84f2d8ae0b7d10c94f7dc: Sending DeleteTablet for 3 replicas of tablet 8298c3ff2b684d23b3ecf34ebf440b4a
I20260618 22:34:12.181747 29484 tablet_service.cc:1514] Processing DeleteTablet for tablet 8298c3ff2b684d23b3ecf34ebf440b4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:34:12 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:42654
I20260618 22:34:12.181754 29751 tablet_service.cc:1514] Processing DeleteTablet for tablet 8298c3ff2b684d23b3ecf34ebf440b4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:34:12 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:53324
I20260618 22:34:12.181806 29618 tablet_service.cc:1514] Processing DeleteTablet for tablet 8298c3ff2b684d23b3ecf34ebf440b4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:34:12 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:51924
I20260618 22:34:12.183780 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 29421
I20260618 22:34:12.196985 29860 tablet_replica.cc:330] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: stopping tablet replica
I20260618 22:34:12.197506 29860 raft_consensus.cc:2229] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:34:12.198057 29860 raft_consensus.cc:2258] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:34:12.200237 29860 ts_tablet_manager.cc:1905] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:34:12.207356 29860 ts_tablet_manager.cc:1918] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:34:12.207531 29860 log.cc:1196] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/8298c3ff2b684d23b3ecf34ebf440b4a
I20260618 22:34:12.208015 29860 ts_tablet_manager.cc:1939] T 8298c3ff2b684d23b3ecf34ebf440b4a P b2ca9d5d1639417a9f41e48c973fc09a: Deleting consensus metadata
I20260618 22:34:12.209836 29346 catalog_manager.cc:4841] TS b2ca9d5d1639417a9f41e48c973fc09a (127.18.150.129:46761): tablet 8298c3ff2b684d23b3ecf34ebf440b4a (table test-table [id=f4716aa270e34f209e3b658168b0a6fa]) successfully deleted
I20260618 22:34:12.211711 29858 tablet_replica.cc:330] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: stopping tablet replica
I20260618 22:34:12.212183 29858 raft_consensus.cc:2229] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:34:12.212819 29858 raft_consensus.cc:2258] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:34:12.214931 29859 tablet_replica.cc:330] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: stopping tablet replica
I20260618 22:34:12.215415 29859 raft_consensus.cc:2229] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:34:12.215986 29859 raft_consensus.cc:2258] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:34:12.216111 29858 ts_tablet_manager.cc:1905] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:34:12.218261 29859 ts_tablet_manager.cc:1905] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 18 22:34:12 dist-test-slave-h7jz krb5kdc[29315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:34:12.224459 29858 ts_tablet_manager.cc:1918] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:34:12.224679 29858 log.cc:1196] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/8298c3ff2b684d23b3ecf34ebf440b4a
I20260618 22:34:12.224931 29859 ts_tablet_manager.cc:1918] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:34:12.225121 29859 log.cc:1196] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/8298c3ff2b684d23b3ecf34ebf440b4a
I20260618 22:34:12.225376 29858 ts_tablet_manager.cc:1939] T 8298c3ff2b684d23b3ecf34ebf440b4a P cd649195a8c2481face8aa07a294cd0e: Deleting consensus metadata
I20260618 22:34:12.225683 29859 ts_tablet_manager.cc:1939] T 8298c3ff2b684d23b3ecf34ebf440b4a P 049663b848c14b7b81ab01448100c7ab: Deleting consensus metadata
I20260618 22:34:12.227243 29344 catalog_manager.cc:4841] TS cd649195a8c2481face8aa07a294cd0e (127.18.150.131:33295): tablet 8298c3ff2b684d23b3ecf34ebf440b4a (table test-table [id=f4716aa270e34f209e3b658168b0a6fa]) successfully deleted
I20260618 22:34:12.227515 29344 catalog_manager.cc:4841] TS 049663b848c14b7b81ab01448100c7ab (127.18.150.130:43319): tablet 8298c3ff2b684d23b3ecf34ebf440b4a (table test-table [id=f4716aa270e34f209e3b658168b0a6fa]) successfully deleted
I20260618 22:34:12.228186 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:12.185336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42662 (local address 127.18.150.129:46761)
0618 22:34:12.185516 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:12.185523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:12.208953 (+ 23430us) server_negotiation.cc:407] Connection header received
0618 22:34:12.209228 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:12.209233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:12.209310 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:12.209379 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:12.211205 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.212215 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.214799 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.215365 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.215470 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:12.220556 (+ 5086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:12.220574 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:12.220593 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:12.220642 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:12.224245 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:12.225126 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:12.225135 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:12.225139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:12.225300 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:12.225753 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:12.225758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:12.225761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:12.226229 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:12.226502 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:12.227762 (+ 1260us) server_negotiation.cc:299] Negotiation successful
0618 22:34:12.227933 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:34:12.229334 29544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:12.370958 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 29421
I20260618 22:34:12.389474 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 29555
Jun 18 22:34:12 dist-test-slave-h7jz krb5kdc[29315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:34:12.407876 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:12.390188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.18.150.130:43319)
0618 22:34:12.390359 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:12.390363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:12.390433 (+ 70us) server_negotiation.cc:407] Connection header received
0618 22:34:12.390614 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:12.390619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:12.390698 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:12.390798 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:12.392374 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.393470 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.395240 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.395866 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.395987 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:12.401070 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:12.401085 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:12.401103 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:12.401150 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:12.404914 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:12.405707 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:12.405717 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:12.405721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:12.405868 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:12.406262 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:12.406268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:12.406271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:12.406744 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:12.407006 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:12.407308 (+ 302us) server_negotiation.cc:299] Negotiation successful
0618 22:34:12.407520 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:34:12.408512 29678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:12.496464 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 29555
I20260618 22:34:12.515646 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 29688
Jun 18 22:34:12 dist-test-slave-h7jz krb5kdc[29315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:34:12.532594 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:12.516392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.18.150.131:33295)
0618 22:34:12.516555 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:12.516559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:12.516595 (+ 36us) server_negotiation.cc:407] Connection header received
0618 22:34:12.516787 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:12.516791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:12.516895 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:12.516976 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:12.518773 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.519540 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.521321 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:12.521722 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:12.521815 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:12.526846 (+ 5031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:12.526863 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:12.526879 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:12.526918 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:12.529839 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:12.530512 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:12.530520 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:12.530522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:12.530705 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:12.531082 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:12.531085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:12.531088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:12.531448 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:12.531746 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:12.532033 (+ 287us) server_negotiation.cc:299] Negotiation successful
0618 22:34:12.532236 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:34:12.533245 29811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:12.651384 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 29688
I20260618 22:34:12.666420 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 29328
I20260618 22:34:12.667510 29390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:12.772002 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 29328
2026-06-18T22:34:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4072 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:34:12 dist-test-slave-h7jz krb5kdc[29868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:34:12 dist-test-slave-h7jz krb5kdc[29868](info): set up 2 sockets
Jun 18 22:34:12 dist-test-slave-h7jz krb5kdc[29868](info): commencing operation
krb5kdc: starting...
W20260618 22:34:13.770555 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:34:13 dist-test-slave-h7jz krb5kdc[29868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:34:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:34:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.150.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.150.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/rangeradmin_127.18.150.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/rangeradmin_127.18.150.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.150.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.150.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/rangerlookup_127.18.150.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/rangerlookup_127.18.150.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.150.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.150.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/HTTP_127.18.150.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/HTTP_127.18.150.148@KRBTEST.COM.keytab.
I20260618 22:34:14.061275 19034 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task8lyoIK/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/postgres -l logfile start
2026-06-18 22:34:16.356 UTC [29898] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-18 22:34:16.356 UTC [29898] LOG: listening on IPv4 address "127.18.150.148", port 45359
2026-06-18 22:34:16.359 UTC [29898] LOG: listening on Unix socket "/tmp/.s.PGSQL.45359"
2026-06-18 22:34:16.379 UTC [29901] LOG: database system was shut down at 2026-06-18 22:34:15 UTC
2026-06-18 22:34:16.382 UTC [29898] LOG: database system is ready to accept connections
I20260618 22:34:17.274158 19034 mini_postgres.cc:96] Postgres bound to 45359
2026-06-18 22:34:17.289 UTC [29909] FATAL: database "slave" does not exist
127.18.150.148:45359 - accepting connections
I20260618 22:34:17.290220 19034 mini_ranger.cc:161] Starting Ranger...
I20260618 22:34:17.313261 19034 mini_ranger.cc:84] Created miniranger Postgres user
I20260618 22:34:17.712342 19034 mini_ranger.cc:87] Created ranger Postgres database
I20260618 22:34:17.712471 19034 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-admin
2026-06-18 22:34:17,988 [I] DB FLAVOR :POSTGRES
2026-06-18 22:34:17,988 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:34:17,988 [I] Checking connection..
2026-06-18 22:34:17,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-18 22:34:18,225 [I] Checking connection passed.
2026-06-18 22:34:18,225 [I] --------- Verifying version history table ---------
2026-06-18 22:34:18,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:34:18,478 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:34:18,478 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:34:18,724 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:34:18,724 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-18 22:34:18,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-18 22:34:18.943 UTC [29973] WARNING: there is no transaction in progress
2026-06-18 22:34:18,949 [I] create_dbversion_catalog.sql file imported successfully
2026-06-18 22:34:18,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:34:19,201 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-18 22:34:19,202 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:34:19,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-18 22:34:19,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-18 22:34:19,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h7jz.c.gcp-upstream.internal','N') ;"
2026-06-18 22:34:19,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:34:20,133 [I] Table x_portal_user does not exist in database ranger
2026-06-18 22:34:20,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:34:20,370 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-18 22:34:20,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:34:20,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:34:20,853 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-18 22:34:20,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-18 22:34:21.772 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.788 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.804 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.819 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.838 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.855 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.909 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.915 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.929 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.939 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.949 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.962 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.972 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.977 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.983 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:21.989 UTC [30081] WARNING: there is no transaction in progress
2026-06-18 22:34:22,515 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-18 22:34:22,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-18 22:34:22,740 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-18 22:34:22,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:34:22,984 [I] Table x_portal_user already exists in database 'ranger'
2026-06-18 22:34:22,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:34:23,238 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-18 22:34:23,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:34:23,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:34:23,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h7jz.c.gcp-upstream.internal';"
2026-06-18 22:34:23,933 [I] CORE_DB_SCHEMA import status has been updated
2026-06-18 22:34:23,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-18 22:34:24,167 [I] DB_PATCHES have already been applied
I20260618 22:34:24.173418 19034 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-admin:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task8lyoIK/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260618 22:34:24.173511 19034 mini_ranger.cc:193] Using host: 127.18.150.148
I20260618 22:34:24.186715 19034 mini_ranger.cc:227] Ranger admin URL: http://127.18.150.148:50517
Jun 18, 2026 10:34:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:34:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:34:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-admin/ews/webapp
Jun 18, 2026 10:34:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 18, 2026 10:34:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 18, 2026 10:34:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-admin/ews/webapp].
Jun 18, 2026 10:34:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260618 22:34:25.208050 19034 mini_ranger.cc:160] Time spent starting Ranger: real 7.918s user 0.000s sys 0.037s
Jun 18, 2026 10:34:43 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 18, 2026 10:34:44 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 18, 2026 10:34:44 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 18, 2026 10:34:44 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 18, 2026 10:34:44 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260618 22:34:45.390843 19034 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.150.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.150.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/rangerkms_127.18.150.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/rangerkms_127.18.150.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.150.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.150.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.150.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/HTTP_127.18.150.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.150.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/HTTP_127.18.150.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 18 22:34:45 dist-test-slave-h7jz krb5kdc[29868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822085, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260618 22:34:45.570566 19034 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260618 22:34:45.589890 19034 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260618 22:34:46.316166 19034 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260618 22:34:46.316323 19034 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-kms
I20260618 22:34:46.316489 19034 mini_ranger_kms.cc:227] Using postgres at 127.18.150.148:45359
2026-06-18 22:34:46,708 [I] DB FLAVOR :POSTGRES
2026-06-18 22:34:46,708 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:34:46,708 [I] Checking connection
2026-06-18 22:34:46,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-18 22:34:46,952 [I] connection success
2026-06-18 22:34:46,953 [I] --------- Verifying Ranger DB tables ---------
2026-06-18 22:34:46,953 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-18 22:34:46,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-18 22:34:47,206 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-18 22:34:47,207 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:34:47,207 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-18 22:34:47,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:45359/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-18 22:34:47,448 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260618 22:34:47.859018 19034 mini_ranger_kms.cc:310] Created kms service
I20260618 22:34:48.041967 19034 mini_ranger_kms.cc:326] Created kudu user
I20260618 22:34:48.082468 19034 mini_ranger_kms.cc:343] Created rangerkms user
Jun 18 22:34:48 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822085, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:34:48.460537 19034 mini_ranger_kms.cc:384] Added ranger policy
I20260618 22:34:48.460836 19034 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-kms:/tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260618 22:34:48.460893 19034 mini_ranger_kms.cc:240] Using host: 127.18.150.148
I20260618 22:34:48.480463 19034 mini_ranger_kms.cc:276] Ranger KMS PID: 30261
I20260618 22:34:48.480620 19034 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.150.148:41159
Jun 18, 2026 10:34:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:34:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:34:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 18, 2026 10:34:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 18, 2026 10:34:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-kms/ews/webapp].
Jun 18, 2026 10:34:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260618 22:34:50.595839 19034 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.025s user 0.088s sys 0.321s
I20260618 22:34:50.596064 19034 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260618 22:34:50.596252 19034 mini_ranger_kms.cc:401] 127.18.150.148:41159/kms/v1/keys
Jun 18 22:34:50 dist-test-slave-h7jz krb5kdc[29868](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781822090, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.150.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 18 22:34:51 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822090, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.150.148@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
Jun 18 22:34:51 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822090, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.150.148@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
Jun 18 22:34:53 dist-test-slave-h7jz krb5kdc[29868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:34:53.781625 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:45891
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:45891
--ranger_config_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:41159/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:34:54.092706 30312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:54.092981 30312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:54.093068 30312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:54.100018 30312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:34:54.100112 30312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:54.100162 30312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:54.100216 30312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:34:54.100261 30312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:34:54.110984 30312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.150.148:41159/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:45891
--ranger_config_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:45891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:54.112144 30312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:54.113379 30312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:54.124361 30318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:54.124341 30320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:54.124821 30317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:54.125039 30312 server_base.cc:1034] running on GCE node
I20260618 22:34:54.125870 30312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:54.127249 30312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:54.128440 30312 hybrid_clock.cc:648] HybridClock initialized: now 1781822094128395 us; error 37 us; skew 500 ppm
I20260618 22:34:54.128753 30312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:34:54 dist-test-slave-h7jz krb5kdc[29868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822094, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:34:54.136072 30312 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:34:54.137506 30312 webserver.cc:460] Webserver started at http://127.18.150.190:38251/ using document root <none> and password file <none>
I20260618 22:34:54.138171 30312 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:54.138283 30312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:54.138551 30312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:34:54 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822094, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:34:54.185811 30312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "a75b6b15c086457dad89898287f8fdb1"
format_stamp: "Formatted at 2026-06-18 22:34:54 on dist-test-slave-h7jz"
server_key: "4270076dd34a299ed7e43a32cc6f2f99"
server_key_iv: "087e9c8fdf8d4943057718aa6acaf043"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:54.186640 30312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "a75b6b15c086457dad89898287f8fdb1"
format_stamp: "Formatted at 2026-06-18 22:34:54 on dist-test-slave-h7jz"
server_key: "4270076dd34a299ed7e43a32cc6f2f99"
server_key_iv: "087e9c8fdf8d4943057718aa6acaf043"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:54.191772 30312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:34:54.207654 30327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:54.209189 30312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:34:54.209369 30312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "a75b6b15c086457dad89898287f8fdb1"
format_stamp: "Formatted at 2026-06-18 22:34:54 on dist-test-slave-h7jz"
server_key: "4270076dd34a299ed7e43a32cc6f2f99"
server_key_iv: "087e9c8fdf8d4943057718aa6acaf043"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:54.209574 30312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:54.228860 30312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:54.232854 30312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:54.233131 30312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:54.251283 30312 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:45891
I20260618 22:34:54.251286 30378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:45891 every 8 connection(s)
I20260618 22:34:54.252673 30312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:34:54.254896 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 30312
I20260618 22:34:54.255107 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:34:54.255910 19034 external_mini_cluster.cc:1428] Setting key 685a2d47f96003b4fdce1018e64505b3
I20260618 22:34:54.256793 30379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:34:54.274052 30379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1: Bootstrap starting.
I20260618 22:34:54.277781 30379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1: Neither blocks nor log segments found. Creating new log.
I20260618 22:34:54.278842 30379 log.cc:824] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1: Log is configured to *not* fsync() on all Append() calls
I20260618 22:34:54.281558 30379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1: No bootstrap required, opened a new log
Jun 18 22:34:54 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:34:54.289137 30379 raft_consensus.cc:348] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75b6b15c086457dad89898287f8fdb1" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 45891 } }
I20260618 22:34:54.289572 30379 raft_consensus.cc:374] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:34:54.289674 30379 raft_consensus.cc:729] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75b6b15c086457dad89898287f8fdb1, State: Initialized, Role: FOLLOWER
I20260618 22:34:54.290293 30379 consensus_queue.cc:260] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75b6b15c086457dad89898287f8fdb1" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 45891 } }
I20260618 22:34:54.290508 30379 raft_consensus.cc:388] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:34:54.290639 30379 raft_consensus.cc:482] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:34:54.290699 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:54.258126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47908 (local address 127.18.150.190:45891)
0618 22:34:54.270567 (+ 12441us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:54.270586 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:54.271007 (+ 421us) server_negotiation.cc:407] Connection header received
0618 22:34:54.273409 (+ 2402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:54.273438 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:54.274015 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:54.274501 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:54.275892 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:54.277808 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:54.279082 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:54.279796 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:54.279986 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:54.283656 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:54.283668 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:54.283683 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:54.283720 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:54.286707 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:54.287362 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:54.287369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:54.287375 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:54.287543 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:54.287912 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:54.287916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:54.287918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:54.288453 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:54.288713 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:54.289156 (+ 443us) server_negotiation.cc:299] Negotiation successful
0618 22:34:54.289563 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12121,"thread_start_us":11882,"threads_started":1}
I20260618 22:34:54.290823 30379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:34:54.293467 30379 raft_consensus.cc:504] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75b6b15c086457dad89898287f8fdb1" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 45891 } }
I20260618 22:34:54.294016 30379 leader_election.cc:302] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a75b6b15c086457dad89898287f8fdb1; no voters:
I20260618 22:34:54.294546 30379 leader_election.cc:288] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:34:54.294657 30384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:34:54.295248 30384 raft_consensus.cc:686] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [term 1 LEADER]: Becoming Leader. State: Replica: a75b6b15c086457dad89898287f8fdb1, State: Running, Role: LEADER
I20260618 22:34:54.295773 30384 consensus_queue.cc:237] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75b6b15c086457dad89898287f8fdb1" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 45891 } }
I20260618 22:34:54.296572 30379 sys_catalog.cc:564] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:34:54.298497 30385 sys_catalog.cc:455] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a75b6b15c086457dad89898287f8fdb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75b6b15c086457dad89898287f8fdb1" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 45891 } } }
I20260618 22:34:54.298478 30386 sys_catalog.cc:455] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a75b6b15c086457dad89898287f8fdb1. Latest consensus state: current_term: 1 leader_uuid: "a75b6b15c086457dad89898287f8fdb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75b6b15c086457dad89898287f8fdb1" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 45891 } } }
I20260618 22:34:54.298883 30386 sys_catalog.cc:458] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [sys.catalog]: This master's current role is: LEADER
I20260618 22:34:54.298890 30385 sys_catalog.cc:458] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1 [sys.catalog]: This master's current role is: LEADER
I20260618 22:34:54.300329 30390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:34:54.304698 30390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:34:54.313572 30390 catalog_manager.cc:1347] Generated new cluster ID: 524809c04ced4e7bb6ba0cfeee57ef2f
I20260618 22:34:54.313690 30390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:34:54.320564 30379 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260618 22:34:54.352480 30390 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:34:54.354569 30390 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:34:54.367198 30390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1: Generated new TSK 0
I20260618 22:34:54.368346 30390 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:34:55.060981 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:41159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:45891
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:34:55 dist-test-slave-h7jz krb5kdc[29868](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781822095, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260618 22:34:55.436334 30424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:55.436749 30424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:55.436952 30424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:55.445165 30424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:55.445298 30424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:55.445480 30424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:34:55.455703 30424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.150.148:41159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:45891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:55.457221 30424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:55.458606 30424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:55.473963 30434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:55.474720 30435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:55.475504 30437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:55.476131 30424 server_base.cc:1034] running on GCE node
I20260618 22:34:55.476765 30424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:55.477725 30424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:55.478946 30424 hybrid_clock.cc:648] HybridClock initialized: now 1781822095478888 us; error 49 us; skew 500 ppm
I20260618 22:34:55.479266 30424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:34:55 dist-test-slave-h7jz krb5kdc[29868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822095, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:34:55.482378 30424 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:34:55.483995 30424 webserver.cc:460] Webserver started at http://127.18.150.129:40533/ using document root <none> and password file <none>
I20260618 22:34:55.484671 30424 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:55.484782 30424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:55.485076 30424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:34:55 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822095, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:34:55.504763 30424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "7ba3e53455ac421680038c56405dc7bb"
format_stamp: "Formatted at 2026-06-18 22:34:55 on dist-test-slave-h7jz"
server_key: "4de34f16e753115bae9e26ae8b687cbd"
server_key_iv: "05294c895614ccc4a69b3a7094fccdf5"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:55.505837 30424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "7ba3e53455ac421680038c56405dc7bb"
format_stamp: "Formatted at 2026-06-18 22:34:55 on dist-test-slave-h7jz"
server_key: "4de34f16e753115bae9e26ae8b687cbd"
server_key_iv: "05294c895614ccc4a69b3a7094fccdf5"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:55.520017 30424 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.007s sys 0.004s
I20260618 22:34:55.562681 30444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:55.564996 30424 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.011s sys 0.008s
I20260618 22:34:55.565305 30424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "7ba3e53455ac421680038c56405dc7bb"
format_stamp: "Formatted at 2026-06-18 22:34:55 on dist-test-slave-h7jz"
server_key: "4de34f16e753115bae9e26ae8b687cbd"
server_key_iv: "05294c895614ccc4a69b3a7094fccdf5"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:55.565685 30424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:55.626705 30424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:55.631430 30424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:55.631711 30424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:55.633147 30424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:55.638219 30424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:55.638391 30424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:55.638586 30424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:55.638705 30424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:55.699841 30424 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:40637
I20260618 22:34:55.699860 30556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:40637 every 8 connection(s)
I20260618 22:34:55.701292 30424 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:34:55.710693 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 30424
I20260618 22:34:55.710889 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:34:55.711686 19034 external_mini_cluster.cc:1428] Setting key 67c9653ccd793b7184b40c84a1425697
Jun 18 22:34:55 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822095, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
I20260618 22:34:55.754479 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:55.704998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:45891 (local address 127.18.150.129:55237)
0618 22:34:55.717336 (+ 12338us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:55.717398 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:55.720749 (+ 3351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:55.722142 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:55.722155 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:55.723148 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:55.724067 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:55.724097 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:55.725751 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:55.725757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:55.727861 (+ 2104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:55.727886 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:55.729367 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:55.729375 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:55.729772 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:55.730704 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:55.730739 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:55.743786 (+ 13047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:55.748132 (+ 4346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:55.748145 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:55.748167 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:55.748879 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:55.750392 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:55.750398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:55.750402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:55.750763 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:55.751512 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:55.751522 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:55.752044 (+ 522us) client_negotiation.cc:769] Sending connection context
0618 22:34:55.752372 (+ 328us) client_negotiation.cc:240] Negotiation successful
0618 22:34:55.752983 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11999,"thread_start_us":11725,"threads_started":1}
I20260618 22:34:55.755951 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:55.705663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:55237 (local address 127.18.150.190:45891)
0618 22:34:55.716705 (+ 11042us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:55.716713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:55.717970 (+ 1257us) server_negotiation.cc:407] Connection header received
0618 22:34:55.721407 (+ 3437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:55.721414 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:55.721518 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:55.721621 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:55.724309 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:55.725557 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:55.728035 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:55.728859 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:55.728979 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:55.744147 (+ 15168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:55.744154 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:55.744160 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:55.744219 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:55.747934 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:55.749055 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:55.749061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:55.749064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:55.749227 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:55.750944 (+ 1717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:55.750950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:55.750954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:55.751346 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:55.755084 (+ 3738us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:55.755262 (+ 178us) server_negotiation.cc:299] Negotiation successful
0618 22:34:55.755530 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10907,"thread_start_us":10795,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
I20260618 22:34:55.757817 30557 heartbeater.cc:347] Connected to a master server at 127.18.150.190:45891
I20260618 22:34:55.758286 30557 heartbeater.cc:464] Registering TS with master...
I20260618 22:34:55.759250 30557 heartbeater.cc:511] Master 127.18.150.190:45891 requested a full tablet report, sending...
I20260618 22:34:55.762161 30344 ts_manager.cc:194] Registered new tserver with Master: 7ba3e53455ac421680038c56405dc7bb (127.18.150.129:40637)
I20260618 22:34:55.764300 30344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:55237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:34:55.793565 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:41159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:45891
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:34:55 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822095, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
Jun 18 22:34:56 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822095, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
W20260618 22:34:56.264281 30563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:56.264784 30563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:56.264992 30563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:56.276790 30563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:56.276940 30563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:56.277146 30563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:34:56.293164 30563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.150.148:41159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:45891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:56.295454 30563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:56.297256 30563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:56.339041 30570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:56.342321 30571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:56.377030 30563 server_base.cc:1034] running on GCE node
W20260618 22:34:56.380316 30573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:56.384555 30563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:56.417271 30563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:56.418797 30563 hybrid_clock.cc:648] HybridClock initialized: now 1781822096418738 us; error 56 us; skew 500 ppm
I20260618 22:34:56.419348 30563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:34:56 dist-test-slave-h7jz krb5kdc[29868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822096, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:34:56.423944 30563 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:34:56.467515 30563 webserver.cc:460] Webserver started at http://127.18.150.130:41363/ using document root <none> and password file <none>
I20260618 22:34:56.468696 30563 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:56.468937 30563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:56.469471 30563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:34:56 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822096, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:34:56.515219 30563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "867ee7772d6a407b9737d28b429c6de7"
format_stamp: "Formatted at 2026-06-18 22:34:56 on dist-test-slave-h7jz"
server_key: "fb7d94dc0d947f09af151cc88d475fed"
server_key_iv: "4404e1e0ea4b204e8a6ef75a6ba2f28d"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:56.516474 30563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "867ee7772d6a407b9737d28b429c6de7"
format_stamp: "Formatted at 2026-06-18 22:34:56 on dist-test-slave-h7jz"
server_key: "fb7d94dc0d947f09af151cc88d475fed"
server_key_iv: "4404e1e0ea4b204e8a6ef75a6ba2f28d"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:56.542374 30563 fs_manager.cc:691] Time spent creating directory manager: real 0.025s user 0.007s sys 0.000s
I20260618 22:34:56.579290 30580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:56.583400 30563 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.002s sys 0.001s
I20260618 22:34:56.583644 30563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "867ee7772d6a407b9737d28b429c6de7"
format_stamp: "Formatted at 2026-06-18 22:34:56 on dist-test-slave-h7jz"
server_key: "fb7d94dc0d947f09af151cc88d475fed"
server_key_iv: "4404e1e0ea4b204e8a6ef75a6ba2f28d"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:56.583917 30563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:56.649858 30563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:56.655441 30563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:56.655848 30563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:56.663368 30563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:56.671869 30563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:56.672070 30563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:56.672269 30563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:56.672430 30563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:56.768471 30557 heartbeater.cc:503] Master 127.18.150.190:45891 was elected leader, sending a full tablet report...
I20260618 22:34:57.099946 30563 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:46229
I20260618 22:34:57.100003 30699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:46229 every 8 connection(s)
I20260618 22:34:57.101549 30563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:34:57.109560 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 30563
I20260618 22:34:57.109742 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:34:57.110250 19034 external_mini_cluster.cc:1428] Setting key d157bef627be5523853f36e2a76d75c7
Jun 18 22:34:57 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822096, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:34:57.135489 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.105525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:33861 (local address 127.18.150.190:45891)
0618 22:34:57.109593 (+ 4068us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:57.109599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:57.112905 (+ 3306us) server_negotiation.cc:407] Connection header received
0618 22:34:57.112984 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:57.112988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:57.113076 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:57.113177 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:57.115433 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.116571 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.122515 (+ 5944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.123140 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.123255 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.127813 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:57.127820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:57.127824 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:57.127864 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:57.131283 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:57.132553 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:57.132559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:57.132563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:57.132708 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:57.133256 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:57.133261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:57.133264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:57.133616 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:57.133766 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:57.134827 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0618 22:34:57.135099 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3946,"thread_start_us":16199,"threads_started":1}
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:34:57.139155 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.104953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:45891 (local address 127.18.150.130:33861)
0618 22:34:57.110101 (+ 5148us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:57.110149 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:57.112477 (+ 2328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:57.113669 (+ 1192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:57.113689 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:57.114332 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:57.115238 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.115266 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.116770 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.116775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.122337 (+ 5562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.122355 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.123602 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.123610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.123891 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.124698 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:57.124727 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:57.127553 (+ 2826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:57.131479 (+ 3926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:57.131488 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:57.131508 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:57.132416 (+ 908us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:57.132850 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:57.132854 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:57.132858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:57.133118 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:57.133870 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:57.133879 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:57.134377 (+ 498us) client_negotiation.cc:769] Sending connection context
0618 22:34:57.136046 (+ 1669us) client_negotiation.cc:240] Negotiation successful
0618 22:34:57.137502 (+ 1456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4831,"thread_start_us":15812,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
I20260618 22:34:57.143443 30700 heartbeater.cc:347] Connected to a master server at 127.18.150.190:45891
I20260618 22:34:57.143795 30700 heartbeater.cc:464] Registering TS with master...
I20260618 22:34:57.144675 30700 heartbeater.cc:511] Master 127.18.150.190:45891 requested a full tablet report, sending...
I20260618 22:34:57.146410 30344 ts_manager.cc:194] Registered new tserver with Master: 867ee7772d6a407b9737d28b429c6de7 (127.18.150.130:46229)
I20260618 22:34:57.147384 30344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:33861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:34:57.168401 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:41159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:45891
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:34:57.482714 30706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:34:57.483007 30706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:34:57.483127 30706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:34:57.490303 30706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:34:57.490424 30706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:34:57.490579 30706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:34:57.500774 30706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.150.148:41159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:45891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:34:57.502099 30706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:34:57.503350 30706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:34:57.515506 30711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:57.515525 30712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:34:57.516220 30714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:34:57.516480 30706 server_base.cc:1034] running on GCE node
I20260618 22:34:57.517156 30706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:34:57.518172 30706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:34:57.519393 30706 hybrid_clock.cc:648] HybridClock initialized: now 1781822097519340 us; error 40 us; skew 500 ppm
I20260618 22:34:57.519730 30706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:34:57 dist-test-slave-h7jz krb5kdc[29868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822097, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:34:57.522889 30706 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:34:57.524274 30706 webserver.cc:460] Webserver started at http://127.18.150.131:39233/ using document root <none> and password file <none>
I20260618 22:34:57.524946 30706 fs_manager.cc:359] Metadata directory not provided
I20260618 22:34:57.525059 30706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:34:57.525419 30706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:34:57 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822097, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:34:57.543558 30706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "072a477ea3f84075826e8daa6c431a8f"
format_stamp: "Formatted at 2026-06-18 22:34:57 on dist-test-slave-h7jz"
server_key: "3894f655174ddb4012ffa12a11761a64"
server_key_iv: "a778e91d90072d914e5a6648c28637cf"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:57.544549 30706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "072a477ea3f84075826e8daa6c431a8f"
format_stamp: "Formatted at 2026-06-18 22:34:57 on dist-test-slave-h7jz"
server_key: "3894f655174ddb4012ffa12a11761a64"
server_key_iv: "a778e91d90072d914e5a6648c28637cf"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:57.549896 30706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:34:57.567535 30721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:57.569465 30706 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:34:57.569662 30706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "072a477ea3f84075826e8daa6c431a8f"
format_stamp: "Formatted at 2026-06-18 22:34:57 on dist-test-slave-h7jz"
server_key: "3894f655174ddb4012ffa12a11761a64"
server_key_iv: "a778e91d90072d914e5a6648c28637cf"
server_key_version: "kuduclusterkey@0"
I20260618 22:34:57.569924 30706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:34:57.590857 30706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:34:57.594622 30706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:34:57.594864 30706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:34:57.595759 30706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:34:57.597054 30706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:34:57.597205 30706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260618 22:34:57.597308 30706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:34:57.597350 30706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:34:57.626192 30706 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:45723
I20260618 22:34:57.626226 30833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:45723 every 8 connection(s)
I20260618 22:34:57.627421 30706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:34:57.629328 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 30706
I20260618 22:34:57.629496 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:34:57.629920 19034 external_mini_cluster.cc:1428] Setting key 12bedc7f3d67f16a38d58b003b5c304e
Jun 18 22:34:57 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822097, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:34:57.646049 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.630013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:34063 (local address 127.18.150.190:45891)
0618 22:34:57.630211 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:57.630216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:57.631431 (+ 1215us) server_negotiation.cc:407] Connection header received
0618 22:34:57.633545 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:57.633549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:57.633637 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:57.633715 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:57.635072 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.635767 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.637228 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.637602 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.637685 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.640732 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:57.640737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:57.640740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:57.640770 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:57.643163 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:57.643943 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:57.643948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:57.643951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:57.644106 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:57.644552 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:57.644556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:57.644558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:57.644799 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:57.644962 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:57.645618 (+ 656us) server_negotiation.cc:299] Negotiation successful
0618 22:34:57.645788 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260618 22:34:57.646924 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.630504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:45891 (local address 127.18.150.131:34063)
0618 22:34:57.631253 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:57.631303 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:57.633300 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:57.633906 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:57.633914 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:57.634372 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:34:57.634947 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.634965 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.635886 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.635889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.637122 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.637133 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.637753 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.637757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.637980 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.638555 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:34:57.638575 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:34:57.640534 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:34:57.643285 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:57.643294 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:57.643310 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:57.643826 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:57.644203 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:34:57.644207 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:34:57.644211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:34:57.644445 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:34:57.644941 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:34:57.644948 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:34:57.645297 (+ 349us) client_negotiation.cc:769] Sending connection context
0618 22:34:57.645518 (+ 221us) client_negotiation.cc:240] Negotiation successful
0618 22:34:57.645872 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":316,"threads_started":1}
I20260618 22:34:57.648015 30834 heartbeater.cc:347] Connected to a master server at 127.18.150.190:45891
I20260618 22:34:57.648283 30834 heartbeater.cc:464] Registering TS with master...
I20260618 22:34:57.648952 30834 heartbeater.cc:511] Master 127.18.150.190:45891 requested a full tablet report, sending...
I20260618 22:34:57.650218 30344 ts_manager.cc:194] Registered new tserver with Master: 072a477ea3f84075826e8daa6c431a8f (127.18.150.131:45723)
I20260618 22:34:57.650972 30344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:34063
I20260618 22:34:57.657260 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:34:57.673358 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.661472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47920 (local address 127.18.150.190:45891)
0618 22:34:57.661632 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:57.661638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:57.662010 (+ 372us) server_negotiation.cc:407] Connection header received
0618 22:34:57.662179 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:57.662184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:57.662259 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:57.662339 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:57.663719 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.664407 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.665684 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.666150 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.666255 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.668434 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:57.668439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:57.668442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:57.668473 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:57.670862 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:57.671516 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:57.671522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:57.671526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:57.671644 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:57.671982 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:57.671987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:57.671991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:57.672295 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:57.672532 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:57.672828 (+ 296us) server_negotiation.cc:299] Negotiation successful
0618 22:34:57.673027 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:34:57.677006 30344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:34:57.684160 30344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:34:57.719484 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.696197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41828 (local address 127.18.150.130:46229)
0618 22:34:57.709095 (+ 12898us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:57.709107 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:57.709141 (+ 34us) server_negotiation.cc:407] Connection header received
0618 22:34:57.709251 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:57.709261 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:57.710280 (+ 1019us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:57.710491 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:34:57.711504 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.714579 (+ 3075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.717208 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.718370 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.718513 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.718684 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:57.718997 (+ 313us) server_negotiation.cc:299] Negotiation successful
0618 22:34:57.719167 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12714,"thread_start_us":12591,"threads_started":1}
I20260618 22:34:57.719483 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.696018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:46229 (local address 127.0.0.1:41828)
0618 22:34:57.706631 (+ 10613us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:57.706687 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:57.706881 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:57.710507 (+ 3626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:57.710515 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:57.710554 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:34:57.711019 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.711033 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.714727 (+ 3694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.714734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.717044 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.717056 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.718536 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.718542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.718714 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.718760 (+ 46us) client_negotiation.cc:769] Sending connection context
0618 22:34:57.718876 (+ 116us) client_negotiation.cc:240] Negotiation successful
0618 22:34:57.719072 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10350,"thread_start_us":19129,"threads_started":1}
I20260618 22:34:57.723470 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.698278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:40637 (local address 127.0.0.1:49768)
0618 22:34:57.715418 (+ 17140us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:57.715443 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:57.715552 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:57.716647 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:57.716653 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:57.716678 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:34:57.717108 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.717122 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.718954 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.718959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.721246 (+ 2287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.721261 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.722958 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.722963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.723072 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.723095 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:34:57.723152 (+ 57us) client_negotiation.cc:240] Negotiation successful
0618 22:34:57.723237 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17013,"thread_start_us":7964,"threads_started":1}
I20260618 22:34:57.723688 30851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.698862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.18.150.129:40637)
0618 22:34:57.715497 (+ 16635us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:57.715507 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:57.715537 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:34:57.715909 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:57.715919 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:57.716181 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:57.716388 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:34:57.717307 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.718813 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.721415 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.722788 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.722980 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.723131 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:57.723209 (+ 78us) server_negotiation.cc:299] Negotiation successful
0618 22:34:57.723361 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16454,"thread_start_us":16685,"threads_started":1}
I20260618 22:34:57.723697 30635 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd35bf56b43465daf95a60b4ba1d7b7 (DEFAULT_TABLE table=test-table [id=a09eb76a2029407ba847cef97ccd74a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:34:57.724774 30849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.696533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58946 (local address 127.18.150.131:45723)
0618 22:34:57.715782 (+ 19249us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:57.715789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:57.715858 (+ 69us) server_negotiation.cc:407] Connection header received
0618 22:34:57.715937 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:57.715942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:57.716180 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:57.716387 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:34:57.717175 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.718748 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.720909 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.722191 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.722374 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.722540 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:57.724286 (+ 1746us) server_negotiation.cc:299] Negotiation successful
0618 22:34:57.724460 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19080,"thread_start_us":19084,"threads_started":1}
I20260618 22:34:57.724857 30848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.696352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:45723 (local address 127.0.0.1:58946)
0618 22:34:57.711601 (+ 15249us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:57.711621 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:57.711734 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:57.716379 (+ 4645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:57.716395 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:57.716421 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:34:57.716818 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.716830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.718880 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.718894 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.720779 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.720793 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.723869 (+ 3076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.723876 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.724002 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.724024 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:34:57.724212 (+ 188us) client_negotiation.cc:240] Negotiation successful
0618 22:34:57.724349 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15114,"mutex_wait_us":1,"thread_start_us":15452,"threads_started":1}
I20260618 22:34:57.727464 30492 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd35bf56b43465daf95a60b4ba1d7b7 (DEFAULT_TABLE table=test-table [id=a09eb76a2029407ba847cef97ccd74a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:34:57.728971 30492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd35bf56b43465daf95a60b4ba1d7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:34:57.728763 30769 tablet_service.cc:1467] Processing CreateTablet for tablet 3dd35bf56b43465daf95a60b4ba1d7b7 (DEFAULT_TABLE table=test-table [id=a09eb76a2029407ba847cef97ccd74a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:34:57.730077 30769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd35bf56b43465daf95a60b4ba1d7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:34:57.730094 30635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dd35bf56b43465daf95a60b4ba1d7b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:34:57.742704 30852 tablet_bootstrap.cc:492] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: Bootstrap starting.
I20260618 22:34:57.745445 30853 tablet_bootstrap.cc:492] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: Bootstrap starting.
I20260618 22:34:57.746237 30852 tablet_bootstrap.cc:654] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: Neither blocks nor log segments found. Creating new log.
I20260618 22:34:57.746876 30854 tablet_bootstrap.cc:492] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: Bootstrap starting.
I20260618 22:34:57.747489 30852 log.cc:824] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: Log is configured to *not* fsync() on all Append() calls
I20260618 22:34:57.748682 30853 tablet_bootstrap.cc:654] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: Neither blocks nor log segments found. Creating new log.
I20260618 22:34:57.749742 30853 log.cc:824] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:34:57.749895 30854 tablet_bootstrap.cc:654] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: Neither blocks nor log segments found. Creating new log.
I20260618 22:34:57.751025 30854 log.cc:824] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: Log is configured to *not* fsync() on all Append() calls
I20260618 22:34:57.752271 30852 tablet_bootstrap.cc:492] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: No bootstrap required, opened a new log
I20260618 22:34:57.752522 30852 ts_tablet_manager.cc:1397] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260618 22:34:57.753750 30854 tablet_bootstrap.cc:492] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: No bootstrap required, opened a new log
I20260618 22:34:57.754019 30854 ts_tablet_manager.cc:1397] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260618 22:34:57.755517 30853 tablet_bootstrap.cc:492] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: No bootstrap required, opened a new log
I20260618 22:34:57.755800 30853 ts_tablet_manager.cc:1397] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260618 22:34:57.762172 30854 raft_consensus.cc:348] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867ee7772d6a407b9737d28b429c6de7" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46229 } } peers { permanent_uuid: "072a477ea3f84075826e8daa6c431a8f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45723 } } peers { permanent_uuid: "7ba3e53455ac421680038c56405dc7bb" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 40637 } }
I20260618 22:34:57.762528 30854 raft_consensus.cc:374] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:34:57.762614 30854 raft_consensus.cc:729] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867ee7772d6a407b9737d28b429c6de7, State: Initialized, Role: FOLLOWER
I20260618 22:34:57.762594 30852 raft_consensus.cc:348] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867ee7772d6a407b9737d28b429c6de7" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46229 } } peers { permanent_uuid: "072a477ea3f84075826e8daa6c431a8f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45723 } } peers { permanent_uuid: "7ba3e53455ac421680038c56405dc7bb" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 40637 } }
I20260618 22:34:57.763018 30852 raft_consensus.cc:374] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:34:57.763108 30852 raft_consensus.cc:729] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba3e53455ac421680038c56405dc7bb, State: Initialized, Role: FOLLOWER
I20260618 22:34:57.763229 30854 consensus_queue.cc:260] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867ee7772d6a407b9737d28b429c6de7" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46229 } } peers { permanent_uuid: "072a477ea3f84075826e8daa6c431a8f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45723 } } peers { permanent_uuid: "7ba3e53455ac421680038c56405dc7bb" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 40637 } }
I20260618 22:34:57.763737 30852 consensus_queue.cc:260] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867ee7772d6a407b9737d28b429c6de7" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46229 } } peers { permanent_uuid: "072a477ea3f84075826e8daa6c431a8f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45723 } } peers { permanent_uuid: "7ba3e53455ac421680038c56405dc7bb" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 40637 } }
I20260618 22:34:57.764287 30700 heartbeater.cc:503] Master 127.18.150.190:45891 was elected leader, sending a full tablet report...
I20260618 22:34:57.764968 30854 ts_tablet_manager.cc:1428] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260618 22:34:57.765746 30853 raft_consensus.cc:348] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867ee7772d6a407b9737d28b429c6de7" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46229 } } peers { permanent_uuid: "072a477ea3f84075826e8daa6c431a8f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45723 } } peers { permanent_uuid: "7ba3e53455ac421680038c56405dc7bb" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 40637 } }
I20260618 22:34:57.766157 30853 raft_consensus.cc:374] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:34:57.766242 30853 raft_consensus.cc:729] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 072a477ea3f84075826e8daa6c431a8f, State: Initialized, Role: FOLLOWER
I20260618 22:34:57.766569 30852 ts_tablet_manager.cc:1428] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260618 22:34:57.766883 30853 consensus_queue.cc:260] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867ee7772d6a407b9737d28b429c6de7" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46229 } } peers { permanent_uuid: "072a477ea3f84075826e8daa6c431a8f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45723 } } peers { permanent_uuid: "7ba3e53455ac421680038c56405dc7bb" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 40637 } }
I20260618 22:34:57.767899 30834 heartbeater.cc:503] Master 127.18.150.190:45891 was elected leader, sending a full tablet report...
I20260618 22:34:57.768703 30853 ts_tablet_manager.cc:1428] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
W20260618 22:34:57.859529 30701 tablet.cc:2369] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:34:57.878863 30835 tablet.cc:2369] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:34:57.896525 30860 raft_consensus.cc:482] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:34:57.896884 30860 raft_consensus.cc:504] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867ee7772d6a407b9737d28b429c6de7" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46229 } } peers { permanent_uuid: "072a477ea3f84075826e8daa6c431a8f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45723 } } peers { permanent_uuid: "7ba3e53455ac421680038c56405dc7bb" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 40637 } }
I20260618 22:34:57.898705 30860 leader_election.cc:288] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 867ee7772d6a407b9737d28b429c6de7 (127.18.150.130:46229), 7ba3e53455ac421680038c56405dc7bb (127.18.150.129:40637)
I20260618 22:34:57.904716 30851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.899308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:43717 (local address 127.18.150.129:40637)
0618 22:34:57.899494 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:57.899500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:57.899520 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:34:57.899593 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:57.899598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:57.899662 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:57.899758 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:34:57.900170 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.901103 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.902916 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.903875 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.903986 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.904039 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:57.904377 (+ 338us) server_negotiation.cc:299] Negotiation successful
0618 22:34:57.904480 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:34:57.904764 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.898944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:40637 (local address 127.18.150.131:43717)
0618 22:34:57.899182 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:57.899204 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:57.899319 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:57.899798 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:57.899802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:57.899844 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:34:57.900020 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.900032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.901251 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.901257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.902768 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.902780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.904023 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.904030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.904217 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.904252 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:34:57.904358 (+ 106us) client_negotiation.cc:240] Negotiation successful
0618 22:34:57.904503 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":1}
I20260618 22:34:57.905354 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.899320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:33979 (local address 127.18.150.130:46229)
0618 22:34:57.899500 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:57.899505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:57.899942 (+ 437us) server_negotiation.cc:407] Connection header received
0618 22:34:57.900064 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:57.900069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:57.900157 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:57.900228 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:34:57.900759 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.901582 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.903505 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.904456 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.904596 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.904663 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:57.905000 (+ 337us) server_negotiation.cc:299] Negotiation successful
0618 22:34:57.905129 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:34:57.905814 30861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.899043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:46229 (local address 127.18.150.131:33979)
0618 22:34:57.899848 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0618 22:34:57.899862 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:34:57.899955 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:34:57.900270 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:34:57.900285 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:34:57.900308 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:34:57.900625 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.900636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.901720 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.901724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.903356 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:34:57.903369 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.904654 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.904660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:34:57.904825 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.904855 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:34:57.905500 (+ 645us) client_negotiation.cc:240] Negotiation successful
0618 22:34:57.905599 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":516,"threads_started":1}
I20260618 22:34:57.905740 30512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd35bf56b43465daf95a60b4ba1d7b7" candidate_uuid: "072a477ea3f84075826e8daa6c431a8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba3e53455ac421680038c56405dc7bb" is_pre_election: true
I20260618 22:34:57.906306 30512 raft_consensus.cc:2454] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 072a477ea3f84075826e8daa6c431a8f in term 0.
I20260618 22:34:57.906733 30655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd35bf56b43465daf95a60b4ba1d7b7" candidate_uuid: "072a477ea3f84075826e8daa6c431a8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867ee7772d6a407b9737d28b429c6de7" is_pre_election: true
I20260618 22:34:57.906906 30722 leader_election.cc:302] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 072a477ea3f84075826e8daa6c431a8f, 7ba3e53455ac421680038c56405dc7bb; no voters:
I20260618 22:34:57.907160 30655 raft_consensus.cc:2454] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 072a477ea3f84075826e8daa6c431a8f in term 0.
I20260618 22:34:57.907258 30860 raft_consensus.cc:2781] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:34:57.907390 30860 raft_consensus.cc:482] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:34:57.907480 30860 raft_consensus.cc:3037] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:34:57.909667 30860 raft_consensus.cc:504] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867ee7772d6a407b9737d28b429c6de7" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46229 } } peers { permanent_uuid: "072a477ea3f84075826e8daa6c431a8f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45723 } } peers { permanent_uuid: "7ba3e53455ac421680038c56405dc7bb" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 40637 } }
I20260618 22:34:57.910259 30860 leader_election.cc:288] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [CANDIDATE]: Term 1 election: Requested vote from peers 867ee7772d6a407b9737d28b429c6de7 (127.18.150.130:46229), 7ba3e53455ac421680038c56405dc7bb (127.18.150.129:40637)
I20260618 22:34:57.910630 30655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd35bf56b43465daf95a60b4ba1d7b7" candidate_uuid: "072a477ea3f84075826e8daa6c431a8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867ee7772d6a407b9737d28b429c6de7"
I20260618 22:34:57.910751 30512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dd35bf56b43465daf95a60b4ba1d7b7" candidate_uuid: "072a477ea3f84075826e8daa6c431a8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba3e53455ac421680038c56405dc7bb"
I20260618 22:34:57.910822 30655 raft_consensus.cc:3037] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:34:57.910966 30512 raft_consensus.cc:3037] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:34:57.912896 30655 raft_consensus.cc:2454] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 072a477ea3f84075826e8daa6c431a8f in term 1.
I20260618 22:34:57.913103 30512 raft_consensus.cc:2454] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 072a477ea3f84075826e8daa6c431a8f in term 1.
I20260618 22:34:57.913240 30723 leader_election.cc:302] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 072a477ea3f84075826e8daa6c431a8f, 867ee7772d6a407b9737d28b429c6de7; no voters:
I20260618 22:34:57.913587 30860 raft_consensus.cc:2781] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:34:57.914062 30860 raft_consensus.cc:686] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 1 LEADER]: Becoming Leader. State: Replica: 072a477ea3f84075826e8daa6c431a8f, State: Running, Role: LEADER
I20260618 22:34:57.914510 30860 consensus_queue.cc:237] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867ee7772d6a407b9737d28b429c6de7" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46229 } } peers { permanent_uuid: "072a477ea3f84075826e8daa6c431a8f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45723 } } peers { permanent_uuid: "7ba3e53455ac421680038c56405dc7bb" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 40637 } }
I20260618 22:34:57.918788 30342 catalog_manager.cc:5495] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 072a477ea3f84075826e8daa6c431a8f (127.18.150.131). New cstate: current_term: 1 leader_uuid: "072a477ea3f84075826e8daa6c431a8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867ee7772d6a407b9737d28b429c6de7" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "072a477ea3f84075826e8daa6c431a8f" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ba3e53455ac421680038c56405dc7bb" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 40637 } health_report { overall_health: UNKNOWN } } }
I20260618 22:34:57.957352 30849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.952191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.18.150.131:45723)
0618 22:34:57.952402 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:57.952409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:57.952486 (+ 77us) server_negotiation.cc:407] Connection header received
0618 22:34:57.952700 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:57.952704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:57.952788 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:57.952915 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:34:57.953290 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.953986 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.955350 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:57.955757 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:57.955864 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:57.956334 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:34:57.956821 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:34:57.956949 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:57.957043 (+ 94us) server_negotiation.cc:299] Negotiation successful
0618 22:34:57.957117 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260618 22:34:57.965068 30655 raft_consensus.cc:1261] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7 [term 1 FOLLOWER]: Refusing update from remote peer 072a477ea3f84075826e8daa6c431a8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:34:57.965087 30512 raft_consensus.cc:1261] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb [term 1 FOLLOWER]: Refusing update from remote peer 072a477ea3f84075826e8daa6c431a8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260618 22:34:57.965278 30558 tablet.cc:2369] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:34:57.966015 30862 consensus_queue.cc:1035] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ba3e53455ac421680038c56405dc7bb" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 40637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:34:57.966370 30860 consensus_queue.cc:1035] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "867ee7772d6a407b9737d28b429c6de7" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:34:57.978286 30867 mvcc.cc:205] Tried to move back new op lower bound from 7298343313255931904 to 7298343313061785600. Current Snapshot: MvccSnapshot[applied={T|T < 7298343313255931904}]
I20260618 22:34:57.990037 30342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:34:57.990279 30342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:34:57.993745 30342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a75b6b15c086457dad89898287f8fdb1: Sending DeleteTablet for 3 replicas of tablet 3dd35bf56b43465daf95a60b4ba1d7b7
I20260618 22:34:57.995024 30769 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd35bf56b43465daf95a60b4ba1d7b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:34:57 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:58946
I20260618 22:34:57.995055 30492 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd35bf56b43465daf95a60b4ba1d7b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:34:57 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:49768
I20260618 22:34:57.995050 30635 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dd35bf56b43465daf95a60b4ba1d7b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:34:57 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:41828
I20260618 22:34:57.995934 30877 tablet_replica.cc:330] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: stopping tablet replica
I20260618 22:34:57.996450 30875 tablet_replica.cc:330] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: stopping tablet replica
I20260618 22:34:57.996933 30875 raft_consensus.cc:2229] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:34:57.997296 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 30424
I20260618 22:34:57.997493 30877 raft_consensus.cc:2229] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:34:57.998085 30875 raft_consensus.cc:2258] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:34:57.998798 30877 raft_consensus.cc:2258] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:34:58.001286 30877 ts_tablet_manager.cc:1905] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:34:58.003639 30875 ts_tablet_manager.cc:1905] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:34:58.007131 30876 tablet_replica.cc:330] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: stopping tablet replica
I20260618 22:34:58.007591 30876 raft_consensus.cc:2229] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:34:58.008193 30876 raft_consensus.cc:2258] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:34:58.010178 30877 ts_tablet_manager.cc:1918] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:34:58.011616 30876 ts_tablet_manager.cc:1905] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 18 22:34:58 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:34:58.012506 30875 ts_tablet_manager.cc:1918] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:34:58.012658 30875 log.cc:1196] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/3dd35bf56b43465daf95a60b4ba1d7b7
I20260618 22:34:58.013152 30875 ts_tablet_manager.cc:1939] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 7ba3e53455ac421680038c56405dc7bb: Deleting consensus metadata
I20260618 22:34:58.015223 30328 catalog_manager.cc:4841] TS 7ba3e53455ac421680038c56405dc7bb (127.18.150.129:40637): tablet 3dd35bf56b43465daf95a60b4ba1d7b7 (table test-table [id=a09eb76a2029407ba847cef97ccd74a0]) successfully deleted
I20260618 22:34:58.015758 30877 log.cc:1196] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/3dd35bf56b43465daf95a60b4ba1d7b7
I20260618 22:34:58.016407 30877 ts_tablet_manager.cc:1939] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 072a477ea3f84075826e8daa6c431a8f: Deleting consensus metadata
I20260618 22:34:58.018124 30330 catalog_manager.cc:4841] TS 072a477ea3f84075826e8daa6c431a8f (127.18.150.131:45723): tablet 3dd35bf56b43465daf95a60b4ba1d7b7 (table test-table [id=a09eb76a2029407ba847cef97ccd74a0]) successfully deleted
I20260618 22:34:58.018265 30876 ts_tablet_manager.cc:1918] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:34:58.018404 30876 log.cc:1196] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/3dd35bf56b43465daf95a60b4ba1d7b7
I20260618 22:34:58.018857 30876 ts_tablet_manager.cc:1939] T 3dd35bf56b43465daf95a60b4ba1d7b7 P 867ee7772d6a407b9737d28b429c6de7: Deleting consensus metadata
I20260618 22:34:58.020560 30329 catalog_manager.cc:4841] TS 867ee7772d6a407b9737d28b429c6de7 (127.18.150.130:46229): tablet 3dd35bf56b43465daf95a60b4ba1d7b7 (table test-table [id=a09eb76a2029407ba847cef97ccd74a0]) successfully deleted
I20260618 22:34:58.020684 30851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:57.999573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49774 (local address 127.18.150.129:40637)
0618 22:34:57.999750 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:57.999756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:58.002203 (+ 2447us) server_negotiation.cc:407] Connection header received
0618 22:34:58.002364 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:58.002368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:58.002447 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:58.002541 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:58.004241 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:58.005324 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:58.007286 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:58.007872 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:58.007965 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:58.014011 (+ 6046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:58.014024 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:58.014041 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:58.014079 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:58.017728 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:58.018632 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:58.018641 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:58.018645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:58.018771 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:58.019152 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:58.019157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:58.019160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:58.019517 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:58.019784 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:58.020108 (+ 324us) server_negotiation.cc:299] Negotiation successful
0618 22:34:58.020331 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:34:58.021562 30552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:58.115571 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 30424
I20260618 22:34:58.136747 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 30563
Jun 18 22:34:58 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:34:58.153687 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:58.137565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.18.150.130:46229)
0618 22:34:58.137748 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:58.137754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:58.137776 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:34:58.137843 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:58.137848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:58.137986 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:58.138104 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:58.139406 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:58.140496 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:58.141845 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:58.142487 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:58.142613 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:58.146732 (+ 4119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:58.146755 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:58.146776 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:58.146820 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:58.150822 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:58.151498 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:58.151509 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:58.151513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:58.151667 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:58.152068 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:58.152073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:58.152076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:58.152513 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:58.152792 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:58.153105 (+ 313us) server_negotiation.cc:299] Negotiation successful
0618 22:34:58.153320 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:34:58.154461 30695 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:58.241304 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 30563
I20260618 22:34:58.259620 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 30706
Jun 18 22:34:58 dist-test-slave-h7jz krb5kdc[29868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:34:58.277983 30849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:34:58.260349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58968 (local address 127.18.150.131:45723)
0618 22:34:58.260531 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0618 22:34:58.260538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:34:58.260616 (+ 78us) server_negotiation.cc:407] Connection header received
0618 22:34:58.260781 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:34:58.260787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:34:58.260880 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:34:58.260995 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:34:58.262613 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:58.263780 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:58.265648 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:34:58.266385 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:34:58.266522 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:34:58.271155 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:34:58.271171 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:34:58.271191 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:34:58.271235 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:34:58.275095 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:58.275846 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:58.275856 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:58.275859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:58.276049 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:34:58.276412 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:34:58.276416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:34:58.276420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:34:58.276910 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:34:58.277212 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0618 22:34:58.277365 (+ 153us) server_negotiation.cc:299] Negotiation successful
0618 22:34:58.277579 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:34:58.278750 30829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:58.398233 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 30706
I20260618 22:34:58.413065 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 30312
I20260618 22:34:58.414196 30374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:34:58.522591 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 30312
I20260618 22:34:58.540743 19034 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260618 22:34:58.875242 19034 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260618 22:34:58.875458 19034 mini_ranger.cc:66] Stopping Ranger...
I20260618 22:34:59.210667 19034 mini_ranger.cc:68] Stopped Ranger
2026-06-18 22:34:59.210 UTC [29898] LOG: received smart shutdown request
2026-06-18 22:34:59.214 UTC [29898] LOG: background worker "logical replication launcher" (PID 29907) exited with exit code 1
2026-06-18 22:34:59.214 UTC [29902] LOG: shutting down
2026-06-18 22:34:59.348 UTC [29898] LOG: database system is shut down
2026-06-18T22:34:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46677 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:34:59 dist-test-slave-h7jz krb5kdc[30891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:34:59 dist-test-slave-h7jz krb5kdc[30891](info): set up 2 sockets
Jun 18 22:34:59 dist-test-slave-h7jz krb5kdc[30891](info): commencing operation
krb5kdc: starting...
W20260618 22:35:00.462056 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:35:00 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:35:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:35:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.150.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.150.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/rangeradmin_127.18.150.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/rangeradmin_127.18.150.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.150.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.150.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/rangerlookup_127.18.150.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/rangerlookup_127.18.150.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.150.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.150.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/HTTP_127.18.150.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/HTTP_127.18.150.148@KRBTEST.COM.keytab.
I20260618 22:35:00.756177 19034 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task8lyoIK/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/postgres -l logfile start
2026-06-18 22:35:02.929 UTC [30921] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-18 22:35:02.929 UTC [30921] LOG: listening on IPv4 address "127.18.150.148", port 53837
2026-06-18 22:35:02.933 UTC [30921] LOG: listening on Unix socket "/tmp/.s.PGSQL.53837"
2026-06-18 22:35:02.952 UTC [30924] LOG: database system was shut down at 2026-06-18 22:35:01 UTC
2026-06-18 22:35:02.955 UTC [30921] LOG: database system is ready to accept connections
I20260618 22:35:03.849720 19034 mini_postgres.cc:96] Postgres bound to 53837
2026-06-18 22:35:03.865 UTC [30932] FATAL: database "slave" does not exist
127.18.150.148:53837 - accepting connections
I20260618 22:35:03.865600 19034 mini_ranger.cc:161] Starting Ranger...
I20260618 22:35:03.889266 19034 mini_ranger.cc:84] Created miniranger Postgres user
I20260618 22:35:04.306905 19034 mini_ranger.cc:87] Created ranger Postgres database
I20260618 22:35:04.307013 19034 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-admin
2026-06-18 22:35:04,575 [I] DB FLAVOR :POSTGRES
2026-06-18 22:35:04,575 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:35:04,575 [I] Checking connection..
2026-06-18 22:35:04,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-18 22:35:04,818 [I] Checking connection passed.
2026-06-18 22:35:04,819 [I] --------- Verifying version history table ---------
2026-06-18 22:35:04,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:35:05,074 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:35:05,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:35:05,328 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:35:05,328 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-18 22:35:05,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-18 22:35:05.573 UTC [30996] WARNING: there is no transaction in progress
2026-06-18 22:35:05,579 [I] create_dbversion_catalog.sql file imported successfully
2026-06-18 22:35:05,579 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:35:05,834 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-18 22:35:05,835 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:35:05,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-18 22:35:06,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-18 22:35:06,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h7jz.c.gcp-upstream.internal','N') ;"
2026-06-18 22:35:06,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:35:06,780 [I] Table x_portal_user does not exist in database ranger
2026-06-18 22:35:06,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:35:07,021 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-18 22:35:07,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:35:07,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:35:07,471 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-18 22:35:07,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-18 22:35:08.421 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.436 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.450 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.464 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.478 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.499 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.566 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.572 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.585 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.595 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.605 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.618 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.627 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.633 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.639 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:08.645 UTC [31104] WARNING: there is no transaction in progress
2026-06-18 22:35:09,254 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-18 22:35:09,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-18 22:35:09,498 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-18 22:35:09,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:35:09,719 [I] Table x_portal_user already exists in database 'ranger'
2026-06-18 22:35:09,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:35:09,955 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-18 22:35:09,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:35:10,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:35:10,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h7jz.c.gcp-upstream.internal';"
2026-06-18 22:35:10,665 [I] CORE_DB_SCHEMA import status has been updated
2026-06-18 22:35:10,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-18 22:35:10,916 [I] DB_PATCHES have already been applied
I20260618 22:35:10.922680 19034 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-admin:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task8lyoIK/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260618 22:35:10.922801 19034 mini_ranger.cc:193] Using host: 127.18.150.148
I20260618 22:35:10.935441 19034 mini_ranger.cc:227] Ranger admin URL: http://127.18.150.148:43403
Jun 18, 2026 10:35:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:35:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:35:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-admin/ews/webapp
Jun 18, 2026 10:35:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 18, 2026 10:35:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 18, 2026 10:35:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-admin/ews/webapp].
Jun 18, 2026 10:35:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260618 22:35:11.950953 19034 mini_ranger.cc:160] Time spent starting Ranger: real 8.085s user 0.005s sys 0.030s
Jun 18, 2026 10:35:30 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 18, 2026 10:35:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 18, 2026 10:35:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 18, 2026 10:35:30 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 18, 2026 10:35:31 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260618 22:35:32.035359 19034 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.150.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.150.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/rangerkms_127.18.150.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.150.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/rangerkms_127.18.150.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.150.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.150.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.150.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/HTTP_127.18.150.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.150.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/HTTP_127.18.150.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 18 22:35:32 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822132, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260618 22:35:32.174213 19034 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260618 22:35:32.191635 19034 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260618 22:35:32.931535 19034 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260618 22:35:32.931653 19034 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-kms
I20260618 22:35:32.931699 19034 mini_ranger_kms.cc:227] Using postgres at 127.18.150.148:53837
2026-06-18 22:35:33,245 [I] DB FLAVOR :POSTGRES
2026-06-18 22:35:33,245 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:35:33,245 [I] Checking connection
2026-06-18 22:35:33,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-18 22:35:33,468 [I] connection success
2026-06-18 22:35:33,468 [I] --------- Verifying Ranger DB tables ---------
2026-06-18 22:35:33,468 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-18 22:35:33,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-18 22:35:33,690 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-18 22:35:33,691 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:35:33,691 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-18 22:35:33,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.150.148:53837/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-18 22:35:33,940 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260618 22:35:34.408396 19034 mini_ranger_kms.cc:310] Created kms service
I20260618 22:35:34.632535 19034 mini_ranger_kms.cc:326] Created kudu user
I20260618 22:35:34.674456 19034 mini_ranger_kms.cc:343] Created rangerkms user
Jun 18 22:35:34 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822132, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:35:35.037057 19034 mini_ranger_kms.cc:384] Added ranger policy
I20260618 22:35:35.037345 19034 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-kms:/tmp/dist-test-task8lyoIK/build/asan/bin/postgresql.jar:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task8lyoIK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260618 22:35:35.037390 19034 mini_ranger_kms.cc:240] Using host: 127.18.150.148
I20260618 22:35:35.054236 19034 mini_ranger_kms.cc:276] Ranger KMS PID: 31284
I20260618 22:35:35.054349 19034 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.150.148:49943
Jun 18, 2026 10:35:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:35:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:35:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 18, 2026 10:35:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 18, 2026 10:35:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-kms/ews/webapp].
Jun 18, 2026 10:35:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260618 22:35:37.107965 19034 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.934s user 0.076s sys 0.249s
I20260618 22:35:37.108176 19034 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260618 22:35:37.108343 19034 mini_ranger_kms.cc:401] 127.18.150.148:49943/kms/v1/keys
Jun 18 22:35:37 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781822137, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.150.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 18 22:35:37 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822137, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.150.148@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
Jun 18 22:35:38 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822137, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.150.148@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
Jun 18 22:35:40 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:35:40.394256 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38279
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:38279
--ranger_config_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:49943/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:35:40.702584 31335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:40.702893 31335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:40.703032 31335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:40.710227 31335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:35:40.710424 31335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:40.710501 31335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:40.710580 31335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:35:40.710641 31335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:35:40.720726 31335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.150.148:49943/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:38279
--ranger_config_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:40.721987 31335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:40.723320 31335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:40.735095 31340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:40.735065 31341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:40.735039 31343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:40.735558 31335 server_base.cc:1034] running on GCE node
I20260618 22:35:40.736266 31335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:40.737556 31335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:40.738737 31335 hybrid_clock.cc:648] HybridClock initialized: now 1781822140738688 us; error 46 us; skew 500 ppm
I20260618 22:35:40.739050 31335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:40 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822140, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:40.741813 31335 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:35:40.743140 31335 webserver.cc:460] Webserver started at http://127.18.150.190:37059/ using document root <none> and password file <none>
I20260618 22:35:40.743762 31335 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:40.743916 31335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:40.744251 31335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:35:40 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822140, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:35:40.799151 31335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "3df55d33ab4047c79887e71849e7ef02"
format_stamp: "Formatted at 2026-06-18 22:35:40 on dist-test-slave-h7jz"
server_key: "a96aff00958c4ddee3fe0178ed17d182"
server_key_iv: "73d4415594b670b9c3aa27f167dda473"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:40.800001 31335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "3df55d33ab4047c79887e71849e7ef02"
format_stamp: "Formatted at 2026-06-18 22:35:40 on dist-test-slave-h7jz"
server_key: "a96aff00958c4ddee3fe0178ed17d182"
server_key_iv: "73d4415594b670b9c3aa27f167dda473"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:40.805379 31335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:35:40.821925 31350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:40.823493 31335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:35:40.823658 31335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "3df55d33ab4047c79887e71849e7ef02"
format_stamp: "Formatted at 2026-06-18 22:35:40 on dist-test-slave-h7jz"
server_key: "a96aff00958c4ddee3fe0178ed17d182"
server_key_iv: "73d4415594b670b9c3aa27f167dda473"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:40.823879 31335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:40.849344 31335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:40.853055 31335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:40.853322 31335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:40.871301 31335 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:38279
I20260618 22:35:40.871304 31401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:38279 every 8 connection(s)
I20260618 22:35:40.872640 31335 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:35:40.876478 31402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:40.876601 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 31335
I20260618 22:35:40.876811 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:35:40.877617 19034 external_mini_cluster.cc:1428] Setting key 8340d52abfa667f4c9d42b52c73dfba8
I20260618 22:35:40.890479 31402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02: Bootstrap starting.
Jun 18 22:35:40 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:40.893653 31402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02: Neither blocks nor log segments found. Creating new log.
I20260618 22:35:40.894721 31402 log.cc:824] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:40.897850 31402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02: No bootstrap required, opened a new log
I20260618 22:35:40.900734 31405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:40.879883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34732 (local address 127.18.150.190:38279)
0618 22:35:40.880851 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:40.880867 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:40.881169 (+ 302us) server_negotiation.cc:407] Connection header received
0618 22:35:40.882970 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:40.883002 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:40.883556 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:40.884016 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:40.885236 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:40.887247 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:40.888484 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:40.889503 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:40.889670 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:40.893504 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:40.893512 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:40.893526 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:40.893565 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:40.896653 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:40.897486 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:40.897492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:40.897498 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:40.897644 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:40.898080 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:40.898084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:40.898086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:40.898581 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:40.898785 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:40.899281 (+ 496us) server_negotiation.cc:299] Negotiation successful
0618 22:35:40.899614 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":383,"threads_started":1}
I20260618 22:35:40.905890 31402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } }
I20260618 22:35:40.906287 31402 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:40.906435 31402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3df55d33ab4047c79887e71849e7ef02, State: Initialized, Role: FOLLOWER
I20260618 22:35:40.907004 31402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } }
I20260618 22:35:40.907197 31402 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:35:40.907333 31402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:35:40.907472 31402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:40.909507 31402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } }
I20260618 22:35:40.910032 31402 leader_election.cc:302] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3df55d33ab4047c79887e71849e7ef02; no voters:
I20260618 22:35:40.910547 31402 leader_election.cc:288] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:35:40.910668 31407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:35:40.911608 31407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 1 LEADER]: Becoming Leader. State: Replica: 3df55d33ab4047c79887e71849e7ef02, State: Running, Role: LEADER
I20260618 22:35:40.912101 31407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } }
I20260618 22:35:40.912549 31402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:35:40.914733 31409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3df55d33ab4047c79887e71849e7ef02. Latest consensus state: current_term: 1 leader_uuid: "3df55d33ab4047c79887e71849e7ef02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } } }
I20260618 22:35:40.915167 31409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:40.914762 31408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3df55d33ab4047c79887e71849e7ef02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } } }
I20260618 22:35:40.915813 31408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:40.916662 31413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:35:40.922735 31413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:35:40.923549 31402 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260618 22:35:40.944424 31413 catalog_manager.cc:1347] Generated new cluster ID: 4a94ef5834534230b04457561e71f593
I20260618 22:35:40.944576 31413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:35:40.972100 31413 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:35:40.974171 31413 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:35:40.991475 31413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02: Generated new TSK 0
I20260618 22:35:40.992663 31413 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:35:41.680816 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:49943/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38279
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:35:41 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781822141, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260618 22:35:42.073596 31447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:42.074004 31447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:42.074170 31447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:42.085525 31447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:42.085664 31447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:42.085891 31447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:35:42.101397 31447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.150.148:49943/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:42.103106 31447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:42.104854 31447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:42.127307 31457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:42.127879 31458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:42.136847 31447 server_base.cc:1034] running on GCE node
W20260618 22:35:42.137265 31460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:42.138260 31447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:42.141098 31447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:42.142339 31447 hybrid_clock.cc:648] HybridClock initialized: now 1781822142142283 us; error 56 us; skew 500 ppm
I20260618 22:35:42.142669 31447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:42 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822142, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:42.159291 31447 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:35:42.161355 31447 webserver.cc:460] Webserver started at http://127.18.150.129:44705/ using document root <none> and password file <none>
I20260618 22:35:42.162169 31447 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:42.162293 31447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:42.162647 31447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:35:42 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822142, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:35:42.181707 31447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "95e5481f9a2e4de182be7c11be4997c3"
format_stamp: "Formatted at 2026-06-18 22:35:42 on dist-test-slave-h7jz"
server_key: "b21172af8bcf6f1eabd27c29fe167a2c"
server_key_iv: "73104da758d4d5fa96ea926422af2dc9"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:42.182817 31447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "95e5481f9a2e4de182be7c11be4997c3"
format_stamp: "Formatted at 2026-06-18 22:35:42 on dist-test-slave-h7jz"
server_key: "b21172af8bcf6f1eabd27c29fe167a2c"
server_key_iv: "73104da758d4d5fa96ea926422af2dc9"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:42.196348 31447 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.005s sys 0.008s
I20260618 22:35:42.224418 31467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:42.226621 31447 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.005s sys 0.000s
I20260618 22:35:42.226905 31447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "95e5481f9a2e4de182be7c11be4997c3"
format_stamp: "Formatted at 2026-06-18 22:35:42 on dist-test-slave-h7jz"
server_key: "b21172af8bcf6f1eabd27c29fe167a2c"
server_key_iv: "73104da758d4d5fa96ea926422af2dc9"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:42.227257 31447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:42.252956 31447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:42.258296 31447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:42.258695 31447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:42.268501 31447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:35:42.270349 31447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:35:42.270438 31447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:42.270586 31447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:35:42.270639 31447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:42.338877 31447 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:39887
I20260618 22:35:42.338932 31579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:39887 every 8 connection(s)
I20260618 22:35:42.340287 31447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:35:42.345389 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 31447
I20260618 22:35:42.345614 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:35:42.346508 19034 external_mini_cluster.cc:1428] Setting key 983b5885a1e5453481f85603d43c5006
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
Jun 18 22:35:42 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822142, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:42.395714 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:42.344033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38279 (local address 127.18.150.129:43529)
0618 22:35:42.370706 (+ 26673us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:42.370774 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:42.373284 (+ 2510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:42.374811 (+ 1527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:42.374825 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:42.375618 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:42.376512 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:42.376538 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:42.379337 (+ 2799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:42.379343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:42.381414 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:42.381430 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:42.382692 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:42.382700 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:42.383012 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:42.383830 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:42.383863 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:42.386647 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:42.390724 (+ 4077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:42.390732 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:42.390752 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:42.391360 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:42.391812 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:42.391817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:42.391821 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:42.392119 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:42.392847 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:42.392857 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:42.393317 (+ 460us) client_negotiation.cc:769] Sending connection context
0618 22:35:42.393648 (+ 331us) client_negotiation.cc:240] Negotiation successful
0618 22:35:42.394260 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26316,"thread_start_us":30060,"threads_started":1}
I20260618 22:35:42.396862 31583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:42.344709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:43529 (local address 127.18.150.190:38279)
0618 22:35:42.370016 (+ 25307us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:42.370023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:42.373764 (+ 3741us) server_negotiation.cc:407] Connection header received
0618 22:35:42.373837 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:42.373842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:42.373990 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:42.374101 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:42.376730 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:42.379138 (+ 2408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:42.381598 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:42.382263 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:42.382386 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:42.386920 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:42.386928 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:42.386932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:42.386984 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:42.390526 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:42.391509 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:42.391516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:42.391520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:42.391675 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:42.392256 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:42.392261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:42.392265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:42.392623 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:42.392755 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:42.396247 (+ 3492us) server_negotiation.cc:299] Negotiation successful
0618 22:35:42.396498 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25101,"thread_start_us":25305,"threads_started":1}
I20260618 22:35:42.399286 31580 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38279
I20260618 22:35:42.399610 31580 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:42.400590 31580 heartbeater.cc:511] Master 127.18.150.190:38279 requested a full tablet report, sending...
I20260618 22:35:42.403412 31367 ts_manager.cc:194] Registered new tserver with Master: 95e5481f9a2e4de182be7c11be4997c3 (127.18.150.129:39887)
I20260618 22:35:42.405406 31367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:43529
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:35:42.407292 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:49943/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38279
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:35:42 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822141, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
Jun 18 22:35:42 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822141, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
W20260618 22:35:42.870518 31586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:42.870927 31586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:42.871119 31586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:42.882472 31586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:42.882594 31586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:42.882769 31586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:35:42.895417 31586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.150.148:49943/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:42.897123 31586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:42.898802 31586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:42.915068 31593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:42.916213 31594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:42.923772 31596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:42.924255 31586 server_base.cc:1034] running on GCE node
I20260618 22:35:42.925002 31586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:42.926043 31586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:42.927335 31586 hybrid_clock.cc:648] HybridClock initialized: now 1781822142927262 us; error 63 us; skew 500 ppm
I20260618 22:35:42.927664 31586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:42 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822142, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:42.930783 31586 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:35:42.932494 31586 webserver.cc:460] Webserver started at http://127.18.150.130:39289/ using document root <none> and password file <none>
I20260618 22:35:42.933218 31586 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:42.933339 31586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:42.933660 31586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:35:42 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822142, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:35:42.951210 31586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "d821faea9ffa409bb9092e51a8bbedb6"
format_stamp: "Formatted at 2026-06-18 22:35:42 on dist-test-slave-h7jz"
server_key: "5058564deb230ba0fc831d0d9b178ab2"
server_key_iv: "ed74e3492b721eeeea0ab7671b9ad93b"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:42.952661 31586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "d821faea9ffa409bb9092e51a8bbedb6"
format_stamp: "Formatted at 2026-06-18 22:35:42 on dist-test-slave-h7jz"
server_key: "5058564deb230ba0fc831d0d9b178ab2"
server_key_iv: "ed74e3492b721eeeea0ab7671b9ad93b"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:42.961753 31586 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260618 22:35:42.981163 31610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:42.983150 31586 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260618 22:35:42.983392 31586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "d821faea9ffa409bb9092e51a8bbedb6"
format_stamp: "Formatted at 2026-06-18 22:35:42 on dist-test-slave-h7jz"
server_key: "5058564deb230ba0fc831d0d9b178ab2"
server_key_iv: "ed74e3492b721eeeea0ab7671b9ad93b"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:42.983722 31586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:43.020944 31586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:43.025336 31586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:43.025629 31586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:43.026647 31586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:35:43.028051 31586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:35:43.028123 31586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:43.028237 31586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:35:43.028319 31586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260618 22:35:43.061175 31586 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:43409
I20260618 22:35:43.061197 31722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:43409 every 8 connection(s)
I20260618 22:35:43.062557 31586 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:35:43.072350 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 31586
I20260618 22:35:43.072522 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:35:43.073144 19034 external_mini_cluster.cc:1428] Setting key 7a727c67c109218ad6a93727b13da098
Jun 18 22:35:43 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822142, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:43.082685 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:43.066106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:38167 (local address 127.18.150.190:38279)
0618 22:35:43.066574 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:43.066581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:43.067056 (+ 475us) server_negotiation.cc:407] Connection header received
0618 22:35:43.068818 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:43.068824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:43.068931 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:43.069027 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:43.070480 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.071206 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.072755 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.073149 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.073265 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:43.077340 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:43.077345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:43.077348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:43.077379 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:43.079848 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:43.080520 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:43.080524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:43.080526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:43.080638 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:43.081018 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:43.081022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:43.081024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:43.081282 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:43.081410 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:43.082080 (+ 670us) server_negotiation.cc:299] Negotiation successful
0618 22:35:43.082310 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":245,"threads_started":1}
I20260618 22:35:43.083775 31726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:43.065917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38279 (local address 127.18.150.130:38167)
0618 22:35:43.066904 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:43.066944 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:43.068552 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:43.069270 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:43.069279 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:43.069703 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:43.070318 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:43.070337 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.071346 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.071349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:43.072618 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:43.072631 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.073419 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.073434 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:43.073639 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:43.075204 (+ 1565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:43.075226 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:43.077144 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:43.079976 (+ 2832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:43.079985 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:43.080003 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:43.080403 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:43.080733 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:43.080738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:43.080741 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:43.080904 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:43.081402 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:43.081409 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:43.081723 (+ 314us) client_negotiation.cc:769] Sending connection context
0618 22:35:43.081980 (+ 257us) client_negotiation.cc:240] Negotiation successful
0618 22:35:43.082466 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":486,"threads_started":1}
I20260618 22:35:43.085491 31723 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38279
I20260618 22:35:43.085806 31723 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:43.086661 31723 heartbeater.cc:511] Master 127.18.150.190:38279 requested a full tablet report, sending...
I20260618 22:35:43.088325 31367 ts_manager.cc:194] Registered new tserver with Master: d821faea9ffa409bb9092e51a8bbedb6 (127.18.150.130:43409)
I20260618 22:35:43.089424 31367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:38167
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:35:43.126014 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:49943/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38279
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:35:43.413427 31580 heartbeater.cc:503] Master 127.18.150.190:38279 was elected leader, sending a full tablet report...
W20260618 22:35:43.442108 31729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:43.442390 31729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:43.442484 31729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:43.449451 31729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:43.449544 31729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:43.449649 31729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:35:43.458945 31729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.150.148:49943/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:43.460117 31729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:43.461305 31729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:43.473214 31737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:43.473269 31734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:43.473300 31735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:43.474067 31729 server_base.cc:1034] running on GCE node
I20260618 22:35:43.474632 31729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:43.475467 31729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:43.476649 31729 hybrid_clock.cc:648] HybridClock initialized: now 1781822143476603 us; error 35 us; skew 500 ppm
I20260618 22:35:43.476926 31729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:43 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822143, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:43.479809 31729 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:35:43.481113 31729 webserver.cc:460] Webserver started at http://127.18.150.131:38871/ using document root <none> and password file <none>
I20260618 22:35:43.481731 31729 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:43.481840 31729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:43.482146 31729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:35:43 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822143, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:35:43.497119 31729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "50fc2a613a4f4fceb88203154add7f77"
format_stamp: "Formatted at 2026-06-18 22:35:43 on dist-test-slave-h7jz"
server_key: "c933fc263c5343ebfdf9e91fbd0b64e4"
server_key_iv: "ca7b5dfea574b362fea01fae17fbbd9f"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:43.497990 31729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "50fc2a613a4f4fceb88203154add7f77"
format_stamp: "Formatted at 2026-06-18 22:35:43 on dist-test-slave-h7jz"
server_key: "c933fc263c5343ebfdf9e91fbd0b64e4"
server_key_iv: "ca7b5dfea574b362fea01fae17fbbd9f"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:43.503459 31729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260618 22:35:43.519156 31744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:43.520664 31729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:35:43.520813 31729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "50fc2a613a4f4fceb88203154add7f77"
format_stamp: "Formatted at 2026-06-18 22:35:43 on dist-test-slave-h7jz"
server_key: "c933fc263c5343ebfdf9e91fbd0b64e4"
server_key_iv: "ca7b5dfea574b362fea01fae17fbbd9f"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:43.521005 31729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:43.547788 31729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:43.551286 31729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:43.551546 31729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:43.552426 31729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:35:43.553687 31729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:35:43.553758 31729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:43.553865 31729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:35:43.553915 31729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:43.581931 31729 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:42821
I20260618 22:35:43.582022 31856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:42821 every 8 connection(s)
I20260618 22:35:43.583141 31729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:35:43.589093 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 31729
I20260618 22:35:43.589294 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:35:43.589898 19034 external_mini_cluster.cc:1428] Setting key e319d60c167969c1d7d3c33597214ece
Jun 18 22:35:43 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822143, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:43.601739 31859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:43.585596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:54987 (local address 127.18.150.190:38279)
0618 22:35:43.586132 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:43.586137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:43.587021 (+ 884us) server_negotiation.cc:407] Connection header received
0618 22:35:43.588772 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:43.588779 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:43.588859 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:43.588970 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:43.590747 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.591578 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.593049 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.593444 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.593557 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:43.596591 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:43.596595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:43.596598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:43.596629 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:43.598966 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:43.599607 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:43.599611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:43.599613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:43.599730 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:43.600112 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:43.600116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:43.600118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:43.600381 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:43.600516 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:43.601163 (+ 647us) server_negotiation.cc:299] Negotiation successful
0618 22:35:43.601380 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":206,"threads_started":1}
I20260618 22:35:43.602676 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:43.586185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38279 (local address 127.18.150.131:54987)
0618 22:35:43.586854 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:43.586897 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:43.588498 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:43.589239 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:43.589252 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:43.589772 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:43.590576 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:43.590597 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.591711 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.591714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:43.592910 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:43.592921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.593650 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.593656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:43.593873 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:43.594493 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:43.594518 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:43.596386 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:43.599084 (+ 2698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:43.599092 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:43.599110 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:43.599490 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:43.599824 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:43.599829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:43.599833 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:43.600005 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:43.600494 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:43.600501 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:43.600812 (+ 311us) client_negotiation.cc:769] Sending connection context
0618 22:35:43.601046 (+ 234us) client_negotiation.cc:240] Negotiation successful
0618 22:35:43.601539 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":251,"threads_started":1}
I20260618 22:35:43.604097 31857 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38279
I20260618 22:35:43.604362 31857 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:43.605126 31857 heartbeater.cc:511] Master 127.18.150.190:38279 requested a full tablet report, sending...
I20260618 22:35:43.606452 31367 ts_manager.cc:194] Registered new tserver with Master: 50fc2a613a4f4fceb88203154add7f77 (127.18.150.131:42821)
I20260618 22:35:43.607285 31367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:54987
I20260618 22:35:43.617249 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:35:43.618918 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 31447
Jun 18 22:35:43 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:35:43.634214 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:43.619672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46426 (local address 127.18.150.129:39887)
0618 22:35:43.620171 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:43.620178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:43.620202 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:35:43.620257 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:43.620263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:43.620464 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:43.620596 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:43.621964 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.622977 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.624333 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.624890 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.624997 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:43.629175 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:43.629186 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:43.629200 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:43.629232 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:43.631875 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:43.632444 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:43.632450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:43.632453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:43.632593 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:43.632890 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:43.632893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:43.632895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:43.633224 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:43.633483 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:43.633666 (+ 183us) server_negotiation.cc:299] Negotiation successful
0618 22:35:43.633851 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":224,"threads_started":1}
I20260618 22:35:43.635360 31575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:43.715140 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 31447
I20260618 22:35:43.728444 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 31586
Jun 18 22:35:43 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:35:43.743816 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:43.729679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36902 (local address 127.18.150.130:43409)
0618 22:35:43.730224 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:43.730233 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:43.730264 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:35:43.730344 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:43.730350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:43.730556 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:43.730680 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:43.731825 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.732788 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.734180 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.734704 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.734792 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:43.738933 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:43.738944 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:43.738958 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:43.738989 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:43.741541 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:43.742167 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:43.742173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:43.742176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:43.742302 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:43.742590 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:43.742593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:43.742595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:43.742902 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:43.743126 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:43.743311 (+ 185us) server_negotiation.cc:299] Negotiation successful
0618 22:35:43.743541 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":282,"threads_started":1}
I20260618 22:35:43.745038 31718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:43.825457 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 31586
I20260618 22:35:43.837800 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 31729
Jun 18 22:35:43 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:35:43.853715 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:43.838874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.18.150.131:42821)
0618 22:35:43.839482 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:43.839489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:43.839517 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:35:43.839580 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:43.839586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:43.839788 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:43.839968 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:43.841249 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.842270 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.843577 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:43.844140 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:43.844250 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:43.848530 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:43.848546 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:43.848561 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:43.848593 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:43.851112 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:43.851755 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:43.851763 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:43.851765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:43.851888 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:43.852221 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:43.852226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:43.852227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:43.852560 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:43.852844 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:43.853144 (+ 300us) server_negotiation.cc:299] Negotiation successful
0618 22:35:43.853384 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":310,"threads_started":1}
I20260618 22:35:43.855124 31852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:43.936082 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 31729
I20260618 22:35:43.947685 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 31335
I20260618 22:35:43.948954 31397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:44.053905 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 31335
I20260618 22:35:44.073794 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38279
--webserver_interface=127.18.150.190
--webserver_port=37059
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:38279
--ranger_config_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:49943/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.150.148:49943/kms with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:35:44.382107 31869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:44.382393 31869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:44.382483 31869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:44.389659 31869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:35:44.389777 31869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:44.389820 31869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:44.389868 31869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:35:44.389919 31869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:35:44.399971 31869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.150.148:49943/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:38279
--ranger_config_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=37059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:44.401211 31869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:44.402529 31869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:44.413486 31874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:44.413520 31875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:44.413555 31877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:44.414644 31869 server_base.cc:1034] running on GCE node
I20260618 22:35:44.415421 31869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:44.416913 31869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:44.418113 31869 hybrid_clock.cc:648] HybridClock initialized: now 1781822144418057 us; error 37 us; skew 500 ppm
I20260618 22:35:44.418490 31869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:44 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822144, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:44.421530 31869 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:35:44.422972 31869 webserver.cc:460] Webserver started at http://127.18.150.190:37059/ using document root <none> and password file <none>
I20260618 22:35:44.423583 31869 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:44.423691 31869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:35:44 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822144, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:35:44.504681 31869 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:35:44.507791 31885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:44.509212 31869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:35:44.509384 31869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "3df55d33ab4047c79887e71849e7ef02"
format_stamp: "Formatted at 2026-06-18 22:35:40 on dist-test-slave-h7jz"
server_key: "a96aff00958c4ddee3fe0178ed17d182"
server_key_iv: "73d4415594b670b9c3aa27f167dda473"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:44.510339 31869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:44.530654 31869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:44.534276 31869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:44.534525 31869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:44.550971 31869 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:38279
I20260618 22:35:44.550994 31936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:38279 every 8 connection(s)
I20260618 22:35:44.552364 31869 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:35:44.555281 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 31869
I20260618 22:35:44.556304 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:49943/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:39887
--local_ip_for_outbound_sockets=127.18.150.129
--tserver_master_addrs=127.18.150.190:38279
--webserver_port=44705
--webserver_interface=127.18.150.129
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.150.148:49943/kms with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:35:44.560753 31937 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:35:44.564901 31937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02: Bootstrap starting.
I20260618 22:35:44.582885 31937 log.cc:824] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:44.591605 31937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:35:44.592090 31937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02: Bootstrap complete.
I20260618 22:35:44.600441 31937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } }
I20260618 22:35:44.601236 31937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3df55d33ab4047c79887e71849e7ef02, State: Initialized, Role: FOLLOWER
I20260618 22:35:44.601809 31937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } }
I20260618 22:35:44.602032 31937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:35:44.602146 31937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:35:44.602312 31937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:35:44.605871 31937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } }
I20260618 22:35:44.606346 31937 leader_election.cc:302] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3df55d33ab4047c79887e71849e7ef02; no voters:
I20260618 22:35:44.606899 31937 leader_election.cc:288] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:35:44.607030 31941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:35:44.608549 31941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [term 2 LEADER]: Becoming Leader. State: Replica: 3df55d33ab4047c79887e71849e7ef02, State: Running, Role: LEADER
I20260618 22:35:44.608933 31937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:35:44.609023 31941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } }
I20260618 22:35:44.611276 31943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3df55d33ab4047c79887e71849e7ef02. Latest consensus state: current_term: 2 leader_uuid: "3df55d33ab4047c79887e71849e7ef02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } } }
I20260618 22:35:44.611637 31943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:44.612170 31942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3df55d33ab4047c79887e71849e7ef02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df55d33ab4047c79887e71849e7ef02" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38279 } } }
I20260618 22:35:44.612452 31942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:44.614902 31947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:35:44.620774 31947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:35:44.622246 31947 catalog_manager.cc:1259] Loaded cluster ID: 4a94ef5834534230b04457561e71f593
I20260618 22:35:44.622393 31947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:35:44.626678 31947 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:35:44.628158 31947 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02: Loaded TSK: 0
I20260618 22:35:44.629767 31947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:35:44.632994 31937 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260618 22:35:44.976648 31939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:44.977001 31939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:44.977182 31939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:44.988253 31939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:44.988466 31939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:44.988716 31939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:35:45.003942 31939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.150.148:49943/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:39887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=44705
--tserver_master_addrs=127.18.150.190:38279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:45.005903 31939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:45.007704 31939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:45.025691 31975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:45.038843 31976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:45.051533 31978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:45.052325 31939 server_base.cc:1034] running on GCE node
I20260618 22:35:45.053185 31939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:45.059149 31939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:45.060509 31939 hybrid_clock.cc:648] HybridClock initialized: now 1781822145060426 us; error 79 us; skew 500 ppm
I20260618 22:35:45.061009 31939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:45 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822145, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:45.065701 31939 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:35:45.098982 31939 webserver.cc:460] Webserver started at http://127.18.150.129:44705/ using document root <none> and password file <none>
I20260618 22:35:45.099964 31939 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:45.100219 31939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:35:45 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822145, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:35:45.132973 31939 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.003s
I20260618 22:35:45.143198 31986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:45.145283 31939 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.003s
I20260618 22:35:45.145548 31939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "95e5481f9a2e4de182be7c11be4997c3"
format_stamp: "Formatted at 2026-06-18 22:35:42 on dist-test-slave-h7jz"
server_key: "b21172af8bcf6f1eabd27c29fe167a2c"
server_key_iv: "73104da758d4d5fa96ea926422af2dc9"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:45.147147 31939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:45.191249 31939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:45.196517 31939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:45.196827 31939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:45.198110 31939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:35:45.199929 31939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:35:45.200094 31939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260618 22:35:45.200219 31939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:35:45.200294 31939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:45.261687 31939 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:39887
I20260618 22:35:45.261731 32099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:39887 every 8 connection(s)
I20260618 22:35:45.263199 31939 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:35:45.270292 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 31939
I20260618 22:35:45.271049 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:49943/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:43409
--local_ip_for_outbound_sockets=127.18.150.130
--tserver_master_addrs=127.18.150.190:38279
--webserver_port=39289
--webserver_interface=127.18.150.130
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.150.148:49943/kms with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:35:45 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822145, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:45.303434 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:45.266185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:53403 (local address 127.18.150.190:38279)
0618 22:35:45.266854 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:45.266873 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:45.267350 (+ 477us) server_negotiation.cc:407] Connection header received
0618 22:35:45.270051 (+ 2701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:45.270075 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:45.270582 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:45.271020 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:45.273441 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:45.275439 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:45.279365 (+ 3926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:45.280436 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:45.280612 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:45.292611 (+ 11999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:45.292624 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:45.292649 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:45.292700 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:45.297454 (+ 4754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:45.298542 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:45.298553 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:45.298561 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:45.298754 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:45.299290 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:45.299295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:45.299299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:45.299931 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:45.300088 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:45.301131 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0618 22:35:45.302051 (+ 920us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":226,"threads_started":1}
I20260618 22:35:45.309826 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:45.266344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38279 (local address 127.18.150.129:53403)
0618 22:35:45.267182 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:45.267224 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:45.269311 (+ 2087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:45.271733 (+ 2422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:45.271745 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:45.272384 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:45.273183 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:45.273211 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:45.275619 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:45.275952 (+ 333us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:45.279170 (+ 3218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:45.279195 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:45.280977 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:45.280984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:45.281238 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:45.286350 (+ 5112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:45.286379 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:45.292231 (+ 5852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:45.297684 (+ 5453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:45.297695 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:45.297713 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:45.298376 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:45.298903 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:45.298907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:45.298910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:45.299168 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:45.300193 (+ 1025us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:45.300201 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:45.300603 (+ 402us) client_negotiation.cc:769] Sending connection context
0618 22:35:45.305314 (+ 4711us) client_negotiation.cc:240] Negotiation successful
0618 22:35:45.306950 (+ 1636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":402,"threads_started":1}
I20260618 22:35:45.313786 32100 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38279
I20260618 22:35:45.314167 32100 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:45.315059 32100 heartbeater.cc:511] Master 127.18.150.190:38279 requested a full tablet report, sending...
W20260618 22:35:45.318441 32100 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:38279 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260618 22:35:45.318677 32100 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:45.318953 32100 heartbeater.cc:511] Master 127.18.150.190:38279 requested a full tablet report, sending...
I20260618 22:35:45.320202 32100 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:45.320544 32100 heartbeater.cc:511] Master 127.18.150.190:38279 requested a full tablet report, sending...
W20260618 22:35:45.321621 32100 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:35:45.631899 32104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:45.632182 32104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:45.632340 32104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:45.640025 32104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:45.640146 32104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:45.640290 32104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:35:45.650527 32104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.150.148:49943/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:43409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=39289
--tserver_master_addrs=127.18.150.190:38279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:45.652084 32104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:45.653375 32104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:45.668973 32119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:45.676025 32118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:45.676491 32121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:45.676694 32104 server_base.cc:1034] running on GCE node
I20260618 22:35:45.677466 32104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:45.678491 32104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:45.679685 32104 hybrid_clock.cc:648] HybridClock initialized: now 1781822145679614 us; error 69 us; skew 500 ppm
I20260618 22:35:45.679986 32104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:45 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822145, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:45.683146 32104 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:35:45.684561 32104 webserver.cc:460] Webserver started at http://127.18.150.130:39289/ using document root <none> and password file <none>
I20260618 22:35:45.685248 32104 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:45.685412 32104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:35:45 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822145, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:35:45.731498 32104 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.004s sys 0.003s
I20260618 22:35:45.743460 32131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:45.744987 32104 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.009s
I20260618 22:35:45.745159 32104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "d821faea9ffa409bb9092e51a8bbedb6"
format_stamp: "Formatted at 2026-06-18 22:35:42 on dist-test-slave-h7jz"
server_key: "5058564deb230ba0fc831d0d9b178ab2"
server_key_iv: "ed74e3492b721eeeea0ab7671b9ad93b"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:45.746387 32104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:45.780560 32104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:45.786477 32104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:45.786900 32104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:45.788273 32104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:35:45.790210 32104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:35:45.790328 32104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:45.790448 32104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:35:45.790529 32104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:35:45 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781822145, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:45.858042 32104 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:43409
I20260618 22:35:45.858104 32243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:43409 every 8 connection(s)
I20260618 22:35:45.859958 32104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:35:45.868712 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 32104
I20260618 22:35:45.869571 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.150.148:49943/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:42821
--local_ip_for_outbound_sockets=127.18.150.131
--tserver_master_addrs=127.18.150.190:38279
--webserver_port=38871
--webserver_interface=127.18.150.131
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.150.148:49943/kms with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:35:45 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822145, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:45.901505 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:45.863245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38279 (local address 127.18.150.130:44735)
0618 22:35:45.881113 (+ 17868us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:45.881164 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:45.883053 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:45.883828 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:45.883838 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:45.884320 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:45.884975 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:45.884994 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:45.886069 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:45.886074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:45.887400 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:45.887415 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:45.888511 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:45.888520 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:45.888829 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:45.889664 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:45.889699 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:45.892466 (+ 2767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:45.896603 (+ 4137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:45.896613 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:45.896632 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:45.897227 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:45.897712 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:45.897718 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:45.897722 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:45.897997 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:45.898806 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:45.898815 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:45.899238 (+ 423us) client_negotiation.cc:769] Sending connection context
0618 22:35:45.899545 (+ 307us) client_negotiation.cc:240] Negotiation successful
0618 22:35:45.900092 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17568,"thread_start_us":17347,"threads_started":1}
I20260618 22:35:45.903431 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:45.863787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:44735 (local address 127.18.150.190:38279)
0618 22:35:45.882040 (+ 18253us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:45.882047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:45.882071 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:35:45.883363 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:45.883368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:45.883454 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:45.883549 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:45.885157 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:45.885898 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:45.887603 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:45.888251 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:45.888381 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:45.892760 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:45.892767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:45.892773 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:45.892838 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:45.896428 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:45.897419 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:45.897425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:45.897429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:45.897581 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:45.898140 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:45.898146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:45.898150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:45.898647 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:45.902638 (+ 3991us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:45.902774 (+ 136us) server_negotiation.cc:299] Negotiation successful
0618 22:35:45.903008 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18083,"thread_start_us":18286,"threads_started":1}
I20260618 22:35:45.904981 32244 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38279
I20260618 22:35:45.905308 32244 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:45.906201 32244 heartbeater.cc:511] Master 127.18.150.190:38279 requested a full tablet report, sending...
I20260618 22:35:45.908438 31902 ts_manager.cc:194] Registered new tserver with Master: d821faea9ffa409bb9092e51a8bbedb6 (127.18.150.130:43409)
I20260618 22:35:45.910660 31902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:44735
W20260618 22:35:46.276547 32248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:46.276904 32248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:46.277103 32248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Jun 18 22:35:46 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822145, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
W20260618 22:35:46.287026 32248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:46.287165 32248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:46.287402 32248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:35:46.301576 32248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.150.148:49943/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:42821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=38871
--tserver_master_addrs=127.18.150.190:38279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:46.303449 32248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:46.305125 32248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:35:46.321817 32100 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:46.322306 32100 heartbeater.cc:511] Master 127.18.150.190:38279 requested a full tablet report, sending...
I20260618 22:35:46.323688 31902 ts_manager.cc:194] Registered new tserver with Master: 95e5481f9a2e4de182be7c11be4997c3 (127.18.150.129:39887)
I20260618 22:35:46.324963 31902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:53403
W20260618 22:35:46.363762 32255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:46.363878 32254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:46.371322 32248 server_base.cc:1034] running on GCE node
W20260618 22:35:46.371562 32257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:46.372341 32248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:46.373601 32248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:46.374886 32248 hybrid_clock.cc:648] HybridClock initialized: now 1781822146374816 us; error 52 us; skew 500 ppm
I20260618 22:35:46.375277 32248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:46 dist-test-slave-h7jz krb5kdc[30891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822146, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:46.379135 32248 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:35:46.380935 32248 webserver.cc:460] Webserver started at http://127.18.150.131:38871/ using document root <none> and password file <none>
I20260618 22:35:46.381740 32248 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:46.381871 32248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:35:46 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822146, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:35:46.404914 32248 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260618 22:35:46.407838 32265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:46.409396 32248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:35:46.409548 32248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "50fc2a613a4f4fceb88203154add7f77"
format_stamp: "Formatted at 2026-06-18 22:35:43 on dist-test-slave-h7jz"
server_key: "c933fc263c5343ebfdf9e91fbd0b64e4"
server_key_iv: "ca7b5dfea574b362fea01fae17fbbd9f"
server_key_version: "kuduclusterkey@0"
I20260618 22:35:46.410619 32248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:46.436693 32248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:46.441854 32248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:46.442176 32248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:46.443389 32248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:35:46.445222 32248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:35:46.445327 32248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:46.445422 32248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:35:46.445487 32248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:46.498018 32248 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:42821
I20260618 22:35:46.498080 32377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:42821 every 8 connection(s)
I20260618 22:35:46.499941 32248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:35:46.505789 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 32248
Jun 18 22:35:46 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822146, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:46.526870 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.502700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:48841 (local address 127.18.150.190:38279)
0618 22:35:46.504043 (+ 1343us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:46.504049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:46.504994 (+ 945us) server_negotiation.cc:407] Connection header received
0618 22:35:46.507511 (+ 2517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:46.507518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:46.507634 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:46.507735 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:46.509964 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.511178 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.513701 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.514513 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.514630 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.519212 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:46.519220 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:46.519226 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:46.519292 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:46.522972 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:46.523944 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:46.523949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:46.523953 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:46.524092 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:46.524613 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:46.524618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:46.524621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:46.524988 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:46.525118 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:46.526136 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0618 22:35:46.526413 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1193,"thread_start_us":1080,"threads_started":1}
I20260618 22:35:46.529512 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.503353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38279 (local address 127.18.150.131:48841)
0618 22:35:46.504411 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:46.504459 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:46.507043 (+ 2584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:46.508216 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:46.508230 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:46.508832 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:46.509693 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.509719 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.511386 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.511391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.513526 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.513540 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.514959 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.514968 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.515284 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.516113 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:46.516147 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:46.518904 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:46.523179 (+ 4275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:46.523188 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:46.523206 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:46.523781 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:46.524215 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:46.524220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:46.524223 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:46.524464 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:46.525196 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:46.525205 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:46.525637 (+ 432us) client_negotiation.cc:769] Sending connection context
0618 22:35:46.527315 (+ 1678us) client_negotiation.cc:240] Negotiation successful
0618 22:35:46.527985 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":515,"threads_started":1}
I20260618 22:35:46.531538 32378 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38279
I20260618 22:35:46.531858 32378 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:46.532750 32378 heartbeater.cc:511] Master 127.18.150.190:38279 requested a full tablet report, sending...
I20260618 22:35:46.534482 31902 ts_manager.cc:194] Registered new tserver with Master: 50fc2a613a4f4fceb88203154add7f77 (127.18.150.131:42821)
I20260618 22:35:46.535607 31902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:48841
I20260618 22:35:46.560086 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.506775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34744 (local address 127.18.150.190:38279)
0618 22:35:46.527238 (+ 20463us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:46.527245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:46.529192 (+ 1947us) server_negotiation.cc:407] Connection header received
0618 22:35:46.529245 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:46.529249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:46.529335 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:46.529434 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:46.531505 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.532535 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.534492 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.535046 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.538034 (+ 2988us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.552265 (+ 14231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:46.552274 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:46.552279 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:46.552329 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:46.555954 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:46.556906 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:46.556913 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:46.556917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:46.557077 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:46.557544 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:46.557549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:46.557553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:46.558005 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:46.558291 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:46.559472 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0618 22:35:46.559704 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":20347,"thread_start_us":22078,"threads_started":1}
I20260618 22:35:46.565443 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:35:46.586748 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.569094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.18.150.190:38279)
0618 22:35:46.569288 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:46.569294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:46.569771 (+ 477us) server_negotiation.cc:407] Connection header received
0618 22:35:46.570078 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:46.570084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:46.570168 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:46.570237 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:46.572369 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.573506 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.575496 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.576163 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.576267 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.579792 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:46.579799 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:46.579804 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:46.579849 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:46.583293 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:46.584187 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:46.584194 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:46.584198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:46.584362 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:46.584835 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:46.584840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:46.584844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:46.585224 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:46.585436 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:46.586076 (+ 640us) server_negotiation.cc:299] Negotiation successful
0618 22:35:46.586375 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:35:46.593197 31902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:35:46.604319 31902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Jun 18 22:35:46 dist-test-slave-h7jz krb5kdc[30891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822145, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for HTTP/127.18.150.148@KRBTEST.COM
I20260618 22:35:46.684124 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.626111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46430 (local address 127.18.150.129:39887)
0618 22:35:46.665484 (+ 39373us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:46.665494 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:46.667953 (+ 2459us) server_negotiation.cc:407] Connection header received
0618 22:35:46.668076 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:46.668086 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:46.668399 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:46.668605 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:35:46.669527 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.671209 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.674890 (+ 3681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.676375 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.676573 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.676767 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:46.683576 (+ 6809us) server_negotiation.cc:299] Negotiation successful
0618 22:35:46.683756 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39172,"thread_start_us":38993,"threads_started":1}
I20260618 22:35:46.685050 32396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.625224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:39887 (local address 127.0.0.1:46430)
0618 22:35:46.667616 (+ 42392us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:46.667639 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:46.667818 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:46.668855 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:46.668861 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:46.668890 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:35:46.669312 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.669323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.672378 (+ 3055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.672382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.674729 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.674745 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.682826 (+ 8081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.682835 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.683047 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.683112 (+ 65us) client_negotiation.cc:769] Sending connection context
0618 22:35:46.683253 (+ 141us) client_negotiation.cc:240] Negotiation successful
0618 22:35:46.684677 (+ 1424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42212,"spinlock_wait_cycles":24832,"thread_start_us":46258,"threads_started":1}
I20260618 22:35:46.686580 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.623365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:42821 (local address 127.0.0.1:34436)
0618 22:35:46.652749 (+ 29384us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:46.652803 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:46.653029 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:46.667552 (+ 14523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:46.667559 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:46.667606 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:35:46.668114 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.668131 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.670546 (+ 2415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.670551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.682819 (+ 12268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.682833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.685799 (+ 2966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.685805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.685987 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.686018 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:35:46.686113 (+ 95us) client_negotiation.cc:240] Negotiation successful
0618 22:35:46.686291 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2427,"thread_start_us":2408,"threads_started":1}
I20260618 22:35:46.687471 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.626981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34436 (local address 127.18.150.131:42821)
0618 22:35:46.666016 (+ 39035us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:46.666025 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:46.666066 (+ 41us) server_negotiation.cc:407] Connection header received
0618 22:35:46.666200 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:46.666210 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:46.666541 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:46.666802 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:35:46.668750 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.670368 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.683050 (+ 12682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.684490 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.685552 (+ 1062us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.686903 (+ 1351us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:46.686984 (+ 81us) server_negotiation.cc:299] Negotiation successful
0618 22:35:46.687135 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38831,"thread_start_us":38652,"threads_started":1}
I20260618 22:35:46.689991 32035 tablet_service.cc:1467] Processing CreateTablet for tablet e23edec16aec4384a955f64b780f439b (DEFAULT_TABLE table=test-table [id=9f33053d0ae34829af6a39ea2068b091]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:35:46.691429 32313 tablet_service.cc:1467] Processing CreateTablet for tablet e23edec16aec4384a955f64b780f439b (DEFAULT_TABLE table=test-table [id=9f33053d0ae34829af6a39ea2068b091]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:35:46.691994 32035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e23edec16aec4384a955f64b780f439b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:46.692860 32313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e23edec16aec4384a955f64b780f439b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:46.698589 32398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.626540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36906 (local address 127.18.150.130:43409)
0618 22:35:46.664919 (+ 38379us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:46.664931 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:46.672080 (+ 7149us) server_negotiation.cc:407] Connection header received
0618 22:35:46.672166 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:46.672172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:46.672383 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:46.672591 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:35:46.676223 (+ 3632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.677714 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.680268 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.690289 (+ 10021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.690479 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.690664 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:46.698105 (+ 7441us) server_negotiation.cc:299] Negotiation successful
0618 22:35:46.698299 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38158,"thread_start_us":38028,"threads_started":1}
I20260618 22:35:46.702006 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.625776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:43409 (local address 127.0.0.1:36906)
0618 22:35:46.671781 (+ 46005us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:46.671802 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:46.671952 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:46.672825 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:46.672830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:46.672854 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:35:46.675996 (+ 3142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.676010 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.678625 (+ 2615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.678630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.680086 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.680101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.697056 (+ 16955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.697065 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.697269 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.697310 (+ 41us) client_negotiation.cc:769] Sending connection context
0618 22:35:46.697436 (+ 126us) client_negotiation.cc:240] Negotiation successful
0618 22:35:46.697597 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45876,"mutex_wait_us":28,"thread_start_us":41448,"threads_started":1}
I20260618 22:35:46.708189 32179 tablet_service.cc:1467] Processing CreateTablet for tablet e23edec16aec4384a955f64b780f439b (DEFAULT_TABLE table=test-table [id=9f33053d0ae34829af6a39ea2068b091]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:35:46.709661 32179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e23edec16aec4384a955f64b780f439b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:46.736419 32402 tablet_bootstrap.cc:492] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: Bootstrap starting.
I20260618 22:35:46.743824 32400 tablet_bootstrap.cc:492] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3: Bootstrap starting.
I20260618 22:35:46.746157 32402 tablet_bootstrap.cc:654] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: Neither blocks nor log segments found. Creating new log.
I20260618 22:35:46.747625 32400 tablet_bootstrap.cc:654] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3: Neither blocks nor log segments found. Creating new log.
I20260618 22:35:46.747776 32402 log.cc:824] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:46.762198 32401 tablet_bootstrap.cc:492] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: Bootstrap starting.
I20260618 22:35:46.764906 32402 tablet_bootstrap.cc:492] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: No bootstrap required, opened a new log
I20260618 22:35:46.765287 32402 ts_tablet_manager.cc:1397] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: Time spent bootstrapping tablet: real 0.029s user 0.015s sys 0.004s
I20260618 22:35:46.767524 32401 tablet_bootstrap.cc:654] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: Neither blocks nor log segments found. Creating new log.
I20260618 22:35:46.768726 32400 log.cc:824] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:46.770588 32401 log.cc:824] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:46.777648 32402 raft_consensus.cc:348] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50fc2a613a4f4fceb88203154add7f77" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42821 } } peers { permanent_uuid: "95e5481f9a2e4de182be7c11be4997c3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 39887 } } peers { permanent_uuid: "d821faea9ffa409bb9092e51a8bbedb6" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43409 } }
I20260618 22:35:46.778330 32402 raft_consensus.cc:374] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:46.778488 32402 raft_consensus.cc:729] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d821faea9ffa409bb9092e51a8bbedb6, State: Initialized, Role: FOLLOWER
I20260618 22:35:46.779404 32402 consensus_queue.cc:260] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50fc2a613a4f4fceb88203154add7f77" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42821 } } peers { permanent_uuid: "95e5481f9a2e4de182be7c11be4997c3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 39887 } } peers { permanent_uuid: "d821faea9ffa409bb9092e51a8bbedb6" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43409 } }
I20260618 22:35:46.782084 32400 tablet_bootstrap.cc:492] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3: No bootstrap required, opened a new log
I20260618 22:35:46.783699 32400 ts_tablet_manager.cc:1397] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3: Time spent bootstrapping tablet: real 0.040s user 0.007s sys 0.003s
I20260618 22:35:46.792851 32401 tablet_bootstrap.cc:492] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: No bootstrap required, opened a new log
I20260618 22:35:46.793322 32401 ts_tablet_manager.cc:1397] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.000s
I20260618 22:35:46.814994 32244 heartbeater.cc:503] Master 127.18.150.190:38279 was elected leader, sending a full tablet report...
I20260618 22:35:46.815330 32400 raft_consensus.cc:348] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50fc2a613a4f4fceb88203154add7f77" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42821 } } peers { permanent_uuid: "95e5481f9a2e4de182be7c11be4997c3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 39887 } } peers { permanent_uuid: "d821faea9ffa409bb9092e51a8bbedb6" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43409 } }
I20260618 22:35:46.815824 32400 raft_consensus.cc:374] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:46.815918 32400 raft_consensus.cc:729] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95e5481f9a2e4de182be7c11be4997c3, State: Initialized, Role: FOLLOWER
I20260618 22:35:46.816586 32400 consensus_queue.cc:260] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50fc2a613a4f4fceb88203154add7f77" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42821 } } peers { permanent_uuid: "95e5481f9a2e4de182be7c11be4997c3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 39887 } } peers { permanent_uuid: "d821faea9ffa409bb9092e51a8bbedb6" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43409 } }
I20260618 22:35:46.818073 32402 ts_tablet_manager.cc:1428] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: Time spent starting tablet: real 0.052s user 0.020s sys 0.021s
I20260618 22:35:46.823509 32401 raft_consensus.cc:348] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50fc2a613a4f4fceb88203154add7f77" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42821 } } peers { permanent_uuid: "95e5481f9a2e4de182be7c11be4997c3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 39887 } } peers { permanent_uuid: "d821faea9ffa409bb9092e51a8bbedb6" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43409 } }
I20260618 22:35:46.824026 32401 raft_consensus.cc:374] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:46.824132 32401 raft_consensus.cc:729] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50fc2a613a4f4fceb88203154add7f77, State: Initialized, Role: FOLLOWER
I20260618 22:35:46.824921 32401 consensus_queue.cc:260] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50fc2a613a4f4fceb88203154add7f77" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42821 } } peers { permanent_uuid: "95e5481f9a2e4de182be7c11be4997c3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 39887 } } peers { permanent_uuid: "d821faea9ffa409bb9092e51a8bbedb6" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43409 } }
I20260618 22:35:46.830691 32100 heartbeater.cc:503] Master 127.18.150.190:38279 was elected leader, sending a full tablet report...
I20260618 22:35:46.831470 32400 ts_tablet_manager.cc:1428] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3: Time spent starting tablet: real 0.047s user 0.010s sys 0.004s
I20260618 22:35:46.836133 32401 ts_tablet_manager.cc:1428] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: Time spent starting tablet: real 0.043s user 0.014s sys 0.000s
I20260618 22:35:46.836524 32378 heartbeater.cc:503] Master 127.18.150.190:38279 was elected leader, sending a full tablet report...
W20260618 22:35:46.868070 32245 tablet.cc:2369] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:35:46.889030 32408 raft_consensus.cc:482] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:35:46.889369 32408 raft_consensus.cc:504] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50fc2a613a4f4fceb88203154add7f77" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42821 } } peers { permanent_uuid: "95e5481f9a2e4de182be7c11be4997c3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 39887 } } peers { permanent_uuid: "d821faea9ffa409bb9092e51a8bbedb6" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43409 } }
I20260618 22:35:46.891552 32408 leader_election.cc:288] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95e5481f9a2e4de182be7c11be4997c3 (127.18.150.129:39887), d821faea9ffa409bb9092e51a8bbedb6 (127.18.150.130:43409)
I20260618 22:35:46.899537 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.892139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:39887 (local address 127.18.150.131:39493)
0618 22:35:46.892523 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:46.892546 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:46.892671 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:46.893234 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:46.893240 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:46.893271 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:35:46.893469 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.893480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.895002 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.895008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.897105 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.897122 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.898872 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.898877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.899049 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.899081 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:35:46.899169 (+ 88us) client_negotiation.cc:240] Negotiation successful
0618 22:35:46.899295 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260618 22:35:46.900244 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.892316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:39493 (local address 127.18.150.129:39887)
0618 22:35:46.892545 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:46.892551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:46.892804 (+ 253us) server_negotiation.cc:407] Connection header received
0618 22:35:46.892885 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:46.892890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:46.892965 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:46.893050 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:35:46.893631 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.894824 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.897316 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.898570 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.898712 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.898782 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:46.899882 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0618 22:35:46.900018 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:35:46.901690 32055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23edec16aec4384a955f64b780f439b" candidate_uuid: "50fc2a613a4f4fceb88203154add7f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95e5481f9a2e4de182be7c11be4997c3" is_pre_election: true
I20260618 22:35:46.902302 32055 raft_consensus.cc:2454] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50fc2a613a4f4fceb88203154add7f77 in term 0.
I20260618 22:35:46.903172 32268 leader_election.cc:302] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50fc2a613a4f4fceb88203154add7f77, 95e5481f9a2e4de182be7c11be4997c3; no voters:
I20260618 22:35:46.903659 32408 raft_consensus.cc:2781] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:35:46.903786 32408 raft_consensus.cc:482] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:35:46.903877 32408 raft_consensus.cc:3037] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:46.910976 32408 raft_consensus.cc:504] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50fc2a613a4f4fceb88203154add7f77" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42821 } } peers { permanent_uuid: "95e5481f9a2e4de182be7c11be4997c3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 39887 } } peers { permanent_uuid: "d821faea9ffa409bb9092e51a8bbedb6" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43409 } }
I20260618 22:35:46.912922 32055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23edec16aec4384a955f64b780f439b" candidate_uuid: "50fc2a613a4f4fceb88203154add7f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95e5481f9a2e4de182be7c11be4997c3"
I20260618 22:35:46.913144 32055 raft_consensus.cc:3037] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:46.916302 32055 raft_consensus.cc:2454] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50fc2a613a4f4fceb88203154add7f77 in term 1.
I20260618 22:35:46.916882 32268 leader_election.cc:302] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50fc2a613a4f4fceb88203154add7f77, 95e5481f9a2e4de182be7c11be4997c3; no voters:
I20260618 22:35:46.925052 32410 raft_consensus.cc:2781] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:35:46.926301 32398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.893238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:47639 (local address 127.18.150.130:43409)
0618 22:35:46.893436 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:46.893442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:46.900820 (+ 7378us) server_negotiation.cc:407] Connection header received
0618 22:35:46.906437 (+ 5617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:46.906444 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:46.906542 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:46.906605 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:35:46.907206 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.908351 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.919377 (+ 11026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.920727 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.920874 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.920972 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:46.925807 (+ 4835us) server_negotiation.cc:299] Negotiation successful
0618 22:35:46.926038 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260618 22:35:46.925433 32408 leader_election.cc:288] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [CANDIDATE]: Term 1 election: Requested vote from peers 95e5481f9a2e4de182be7c11be4997c3 (127.18.150.129:39887), d821faea9ffa409bb9092e51a8bbedb6 (127.18.150.130:43409)
I20260618 22:35:46.927142 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.892632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:43409 (local address 127.18.150.131:47639)
0618 22:35:46.900726 (+ 8094us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:46.900742 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:46.906231 (+ 5489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:46.906804 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:46.906809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:46.906835 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:35:46.907028 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.907039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.908539 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.908545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.911859 (+ 3314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:46.911874 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.921213 (+ 9339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.921224 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:46.925573 (+ 4349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.925621 (+ 48us) client_negotiation.cc:769] Sending connection context
0618 22:35:46.926659 (+ 1038us) client_negotiation.cc:240] Negotiation successful
0618 22:35:46.926823 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8003,"thread_start_us":17445,"threads_started":1}
I20260618 22:35:46.932616 32199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23edec16aec4384a955f64b780f439b" candidate_uuid: "50fc2a613a4f4fceb88203154add7f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d821faea9ffa409bb9092e51a8bbedb6" is_pre_election: true
I20260618 22:35:46.933224 32199 raft_consensus.cc:2454] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50fc2a613a4f4fceb88203154add7f77 in term 0.
I20260618 22:35:46.933791 32198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23edec16aec4384a955f64b780f439b" candidate_uuid: "50fc2a613a4f4fceb88203154add7f77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d821faea9ffa409bb9092e51a8bbedb6"
I20260618 22:35:46.934146 32198 raft_consensus.cc:3037] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:46.937320 32198 raft_consensus.cc:2454] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50fc2a613a4f4fceb88203154add7f77 in term 1.
I20260618 22:35:46.942222 32410 raft_consensus.cc:686] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 1 LEADER]: Becoming Leader. State: Replica: 50fc2a613a4f4fceb88203154add7f77, State: Running, Role: LEADER
I20260618 22:35:46.942925 32410 consensus_queue.cc:237] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50fc2a613a4f4fceb88203154add7f77" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42821 } } peers { permanent_uuid: "95e5481f9a2e4de182be7c11be4997c3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 39887 } } peers { permanent_uuid: "d821faea9ffa409bb9092e51a8bbedb6" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43409 } }
I20260618 22:35:46.961169 31902 catalog_manager.cc:5495] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50fc2a613a4f4fceb88203154add7f77 (127.18.150.131). New cstate: current_term: 1 leader_uuid: "50fc2a613a4f4fceb88203154add7f77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50fc2a613a4f4fceb88203154add7f77" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 42821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95e5481f9a2e4de182be7c11be4997c3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 39887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d821faea9ffa409bb9092e51a8bbedb6" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43409 } health_report { overall_health: UNKNOWN } } }
I20260618 22:35:46.998867 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:46.993419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34448 (local address 127.18.150.131:42821)
0618 22:35:46.993612 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:46.993619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:46.993796 (+ 177us) server_negotiation.cc:407] Connection header received
0618 22:35:46.994051 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:46.994055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:46.994138 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:46.994266 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:35:46.994658 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.995367 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.996754 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:46.997158 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:46.997263 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:46.997768 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:35:46.998314 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:35:46.998450 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:46.998528 (+ 78us) server_negotiation.cc:299] Negotiation successful
0618 22:35:46.998608 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260618 22:35:47.002990 32379 tablet.cc:2369] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:35:47.007282 32198 raft_consensus.cc:1261] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6 [term 1 FOLLOWER]: Refusing update from remote peer 50fc2a613a4f4fceb88203154add7f77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:35:47.007282 32055 raft_consensus.cc:1261] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3 [term 1 FOLLOWER]: Refusing update from remote peer 50fc2a613a4f4fceb88203154add7f77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:35:47.008322 32410 consensus_queue.cc:1035] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95e5481f9a2e4de182be7c11be4997c3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 39887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:35:47.008678 32411 consensus_queue.cc:1035] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d821faea9ffa409bb9092e51a8bbedb6" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 43409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:35:47.015734 32101 tablet.cc:2369] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:35:47.036669 32421 mvcc.cc:205] Tried to move back new op lower bound from 7298343514130927616 to 7298343513884053504. Current Snapshot: MvccSnapshot[applied={T|T < 7298343514130927616}]
I20260618 22:35:47.044013 32423 mvcc.cc:205] Tried to move back new op lower bound from 7298343514130927616 to 7298343513884053504. Current Snapshot: MvccSnapshot[applied={T|T < 7298343514130927616}]
I20260618 22:35:47.081602 31901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:35:47.081919 31901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:35:47.086462 31901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3df55d33ab4047c79887e71849e7ef02: Sending DeleteTablet for 3 replicas of tablet e23edec16aec4384a955f64b780f439b
I20260618 22:35:47.087687 32313 tablet_service.cc:1514] Processing DeleteTablet for tablet e23edec16aec4384a955f64b780f439b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:35:47 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:34436
I20260618 22:35:47.088277 32035 tablet_service.cc:1514] Processing DeleteTablet for tablet e23edec16aec4384a955f64b780f439b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:35:47 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:46430
I20260618 22:35:47.089509 32179 tablet_service.cc:1514] Processing DeleteTablet for tablet e23edec16aec4384a955f64b780f439b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:35:47 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:36906
I20260618 22:35:47.090868 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 31939
I20260618 22:35:47.122185 32431 tablet_replica.cc:330] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3: stopping tablet replica
I20260618 22:35:47.123202 32432 tablet_replica.cc:330] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: stopping tablet replica
I20260618 22:35:47.123663 32432 raft_consensus.cc:2229] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:35:47.124185 32432 raft_consensus.cc:2258] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:35:47.130067 32430 tablet_replica.cc:330] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: stopping tablet replica
I20260618 22:35:47.130514 32430 raft_consensus.cc:2229] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:35:47.136564 32432 ts_tablet_manager.cc:1905] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:35:47.137908 32430 raft_consensus.cc:2258] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:35:47.140202 32430 ts_tablet_manager.cc:1905] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:35:47.150164 32431 raft_consensus.cc:2229] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:35:47.153565 32431 raft_consensus.cc:2258] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:35:47.155761 32431 ts_tablet_manager.cc:1905] T e23edec16aec4384a955f64b780f439b P 95e5481f9a2e4de182be7c11be4997c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:35:47.156508 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:47.092141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46434 (local address 127.18.150.129:39887)
0618 22:35:47.092319 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:47.092325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:47.119985 (+ 27660us) server_negotiation.cc:407] Connection header received
0618 22:35:47.120269 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:47.120275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:47.120357 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:47.120437 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:47.125760 (+ 5323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:47.126990 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:47.138445 (+ 11455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:47.139124 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:47.139255 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:47.148485 (+ 9230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:47.148503 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:47.148522 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:47.148568 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:47.152698 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:47.153861 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:47.153871 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:47.153874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:47.154058 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:47.154608 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:47.154613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:47.154617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:47.155047 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:47.155332 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:47.155921 (+ 589us) server_negotiation.cc:299] Negotiation successful
0618 22:35:47.156133 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:35:47.157900 32095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:47.160132 32432 ts_tablet_manager.cc:1918] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:35:47.160331 32432 log.cc:1196] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/e23edec16aec4384a955f64b780f439b
I20260618 22:35:47.160853 32432 ts_tablet_manager.cc:1939] T e23edec16aec4384a955f64b780f439b P d821faea9ffa409bb9092e51a8bbedb6: Deleting consensus metadata
I20260618 22:35:47.162839 31886 catalog_manager.cc:4841] TS d821faea9ffa409bb9092e51a8bbedb6 (127.18.150.130:43409): tablet e23edec16aec4384a955f64b780f439b (table test-table [id=9f33053d0ae34829af6a39ea2068b091]) successfully deleted
I20260618 22:35:47.164489 32430 ts_tablet_manager.cc:1918] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:35:47.164685 32430 log.cc:1196] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/e23edec16aec4384a955f64b780f439b
I20260618 22:35:47.165220 32430 ts_tablet_manager.cc:1939] T e23edec16aec4384a955f64b780f439b P 50fc2a613a4f4fceb88203154add7f77: Deleting consensus metadata
I20260618 22:35:47.166922 31889 catalog_manager.cc:4841] TS 50fc2a613a4f4fceb88203154add7f77 (127.18.150.131:42821): tablet e23edec16aec4384a955f64b780f439b (table test-table [id=9f33053d0ae34829af6a39ea2068b091]) successfully deleted
I20260618 22:35:47.332875 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 31939
W20260618 22:35:47.352726 31888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.150.129:39887 (error 108)
I20260618 22:35:47.359364 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 32104
I20260618 22:35:47.372817 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:47.365980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:39887 (local address 127.0.0.1:46440)
0618 22:35:47.372445 (+ 6465us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:47.372602 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6313,"threads_started":1}
W20260618 22:35:47.373253 31888 catalog_manager.cc:4584] TS 95e5481f9a2e4de182be7c11be4997c3 (127.18.150.129:39887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e23edec16aec4384a955f64b780f439b: Network error: Client connection negotiation failed: client connection to 127.18.150.129:39887: connect: Connection refused (error 111)
I20260618 22:35:47.382678 32398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:47.360853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36918 (local address 127.18.150.130:43409)
0618 22:35:47.361016 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:47.361021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:47.361039 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:35:47.361098 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:47.361103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:47.361168 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:47.361248 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:47.363004 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:47.363995 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:47.365824 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:47.366464 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:47.366580 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:47.374595 (+ 8015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:47.374615 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:47.374636 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:47.374681 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:47.378485 (+ 3804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:47.379393 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:47.379403 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:47.379407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:47.379592 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:47.380039 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:47.380044 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:47.380048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:47.380521 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:47.380805 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:47.382097 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0618 22:35:47.382305 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:35:47.383328 32239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:47.427050 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:47.426512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:39887 (local address 127.0.0.1:46446)
0618 22:35:47.426776 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:47.426882 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260618 22:35:47.428164 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:47.427739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:39887 (local address 127.0.0.1:46458)
0618 22:35:47.427934 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:47.428012 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260618 22:35:47.488739 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:47.488170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:39887 (local address 127.0.0.1:46462)
0618 22:35:47.488453 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:47.488562 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260618 22:35:47.489924 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:47.489487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:39887 (local address 127.0.0.1:46470)
0618 22:35:47.489676 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:47.489754 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260618 22:35:47.535126 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 32104
I20260618 22:35:47.555203 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 32248
I20260618 22:35:47.570581 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:47.570036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:39887 (local address 127.0.0.1:46472)
0618 22:35:47.570305 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:47.570410 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260618 22:35:47.571957 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:47.571460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:39887 (local address 127.0.0.1:46474)
0618 22:35:47.571712 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:47.571796 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260618 22:35:47.585446 32438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:47.556028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34462 (local address 127.18.150.131:42821)
0618 22:35:47.563813 (+ 7785us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:47.563819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:47.563846 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:35:47.563913 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:47.563918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:47.564008 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:47.566094 (+ 2086us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:47.566291 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:47.567412 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:47.569234 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:47.569843 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:47.577040 (+ 7197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:47.578028 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:47.578045 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:47.578065 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:47.578107 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:47.582101 (+ 3994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:47.582980 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:47.582990 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:47.582994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:47.583194 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:47.583618 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:47.583624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:47.583627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:47.584137 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:47.584447 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:47.584751 (+ 304us) server_negotiation.cc:299] Negotiation successful
0618 22:35:47.584994 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7649,"thread_start_us":8104,"threads_started":1}
I20260618 22:35:47.586647 32373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:47.706785 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 32248
I20260618 22:35:47.729364 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 31869
I20260618 22:35:47.730633 31932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:47.846421 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:47.845836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:39887 (local address 127.0.0.1:46482)
0618 22:35:47.846129 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:47.846233 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260618 22:35:47.846771 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 31869
I20260618 22:35:47.871783 19034 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260618 22:35:48.206913 19034 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260618 22:35:48.207113 19034 mini_ranger.cc:66] Stopping Ranger...
I20260618 22:35:48.542013 19034 mini_ranger.cc:68] Stopped Ranger
2026-06-18 22:35:48.542 UTC [30921] LOG: received smart shutdown request
2026-06-18 22:35:48.544 UTC [30921] LOG: background worker "logical replication launcher" (PID 30930) exited with exit code 1
2026-06-18 22:35:48.545 UTC [30925] LOG: shutting down
2026-06-18 22:35:48.683 UTC [30921] LOG: database system is shut down
2026-06-18T22:35:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49333 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:35:48 dist-test-slave-h7jz krb5kdc[32449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:35:48 dist-test-slave-h7jz krb5kdc[32449](info): set up 2 sockets
Jun 18 22:35:48 dist-test-slave-h7jz krb5kdc[32449](info): commencing operation
krb5kdc: starting...
W20260618 22:35:49.790010 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.969s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:35:49 dist-test-slave-h7jz krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:35:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:35:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:35:49.998869 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:41047
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:33111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:41047,127.18.150.189:40979,127.18.150.188:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:35:50.316037 32462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:50.316320 32462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:50.316447 32462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:50.323699 32462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:35:50.323815 32462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:50.323889 32462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:50.323959 32462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:35:50.324013 32462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:35:50.333915 32462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:33111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:41047,127.18.150.189:40979,127.18.150.188:39071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:41047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:50.335289 32462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:50.336673 32462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:50.348433 32467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:50.348405 32470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:50.348367 32468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:50.349367 32462 server_base.cc:1034] running on GCE node
I20260618 22:35:50.350119 32462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:50.351439 32462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:50.352651 32462 hybrid_clock.cc:648] HybridClock initialized: now 1781822150352607 us; error 30 us; skew 500 ppm
I20260618 22:35:50.352973 32462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:50 dist-test-slave-h7jz krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822150, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:50.355995 32462 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:35:50.357367 32462 webserver.cc:460] Webserver started at http://127.18.150.190:32899/ using document root <none> and password file <none>
I20260618 22:35:50.358069 32462 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:50.358175 32462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:50.358421 32462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:35:50.361232 32462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "a2da061c09aa45e88de430ab923e5e59"
format_stamp: "Formatted at 2026-06-18 22:35:50 on dist-test-slave-h7jz"
server_key: "7e5f9de5c8b81bf908da7de1ffcfe8f5"
server_key_iv: "7014e5be8ad664270553021f415f7766"
server_key_version: "encryptionkey@0"
I20260618 22:35:50.361992 32462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "a2da061c09aa45e88de430ab923e5e59"
format_stamp: "Formatted at 2026-06-18 22:35:50 on dist-test-slave-h7jz"
server_key: "7e5f9de5c8b81bf908da7de1ffcfe8f5"
server_key_iv: "7014e5be8ad664270553021f415f7766"
server_key_version: "encryptionkey@0"
I20260618 22:35:50.366881 32462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260618 22:35:50.370246 32477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:50.371575 32462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:35:50.371724 32462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "a2da061c09aa45e88de430ab923e5e59"
format_stamp: "Formatted at 2026-06-18 22:35:50 on dist-test-slave-h7jz"
server_key: "7e5f9de5c8b81bf908da7de1ffcfe8f5"
server_key_iv: "7014e5be8ad664270553021f415f7766"
server_key_version: "encryptionkey@0"
I20260618 22:35:50.371897 32462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:50.383798 32462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:50.387706 32462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:50.387967 32462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:50.404744 32462 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:41047
I20260618 22:35:50.404762 32528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:41047 every 8 connection(s)
I20260618 22:35:50.406107 32462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:35:50.408653 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 32462
I20260618 22:35:50.408875 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:35:50.409686 19034 external_mini_cluster.cc:1428] Setting key 5475b7cfe29231d322f057cbd5e5c2df
I20260618 22:35:50.410171 32529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:50.418651 32529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.150.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.189@KRBTEST.COM" created.
Jun 18 22:35:50 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822150, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:50.441610 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.422742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41324 (local address 127.18.150.190:41047)
0618 22:35:50.423351 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:50.423360 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:50.423403 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:35:50.425231 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:50.425237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:50.425469 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:50.425625 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:50.427146 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.428394 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.430446 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.431019 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.431142 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:50.434419 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:50.434429 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:50.434443 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:50.434482 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:50.437590 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:50.438323 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:50.438332 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:50.438335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:50.438509 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:50.438853 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:50.438857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:50.438859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:50.439477 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:50.439683 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:50.440295 (+ 612us) server_negotiation.cc:299] Negotiation successful
0618 22:35:50.440592 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":456,"thread_start_us":319,"threads_started":1}
I20260618 22:35:50.441665 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.421640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:41047 (local address 127.0.0.1:41324)
0618 22:35:50.422463 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.422517 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:50.424838 (+ 2321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:50.425697 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:50.425704 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:50.426468 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:50.426950 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:50.426973 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.428540 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.428545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:50.430312 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:50.430333 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.431217 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.431222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:50.431361 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:50.431964 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:50.431991 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:50.434190 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:50.437726 (+ 3536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:50.437736 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:50.437745 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:50.438188 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:50.438611 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:50.438614 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:50.438617 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:50.438763 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:50.439595 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:50.439600 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:50.439834 (+ 234us) client_negotiation.cc:769] Sending connection context
0618 22:35:50.440111 (+ 277us) client_negotiation.cc:240] Negotiation successful
0618 22:35:50.440752 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":132,"thread_start_us":334,"threads_started":1}
I20260618 22:35:50.444815 32529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:35:50.445725 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.445378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51176)
0618 22:35:50.445531 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.445628 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260618 22:35:50.445986 32478 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
I20260618 22:35:50.447373 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.447068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51182)
0618 22:35:50.447194 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.447259 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260618 22:35:50.447542 32529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:40979: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab.
Entry for principal kudu/127.18.150.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab.
I20260618 22:35:50.459092 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.189:40979
--webserver_interface=127.18.150.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab
--principal=kudu/127.18.150.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:33111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:41047,127.18.150.189:40979,127.18.150.188:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:35:50.496872 32529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } attempt: 1
I20260618 22:35:50.498021 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.497556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51190)
0618 22:35:50.497736 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.497854 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260618 22:35:50.498936 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.498622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51204)
0618 22:35:50.498743 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.498810 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260618 22:35:50.499130 32529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:40979: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
I20260618 22:35:50.567360 32529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } attempt: 2
I20260618 22:35:50.568604 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.568141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51214)
0618 22:35:50.568365 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.568454 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260618 22:35:50.569700 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.569293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51220)
0618 22:35:50.569431 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.569509 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260618 22:35:50.569962 32529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:40979: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
I20260618 22:35:50.661237 32529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } attempt: 3
I20260618 22:35:50.662467 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.662043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51222)
0618 22:35:50.662234 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.662338 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260618 22:35:50.663470 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.663098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51238)
0618 22:35:50.663242 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.663314 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260618 22:35:50.663645 32529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:40979: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
W20260618 22:35:50.769114 32536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:50.769405 32536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:50.769537 32536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:50.776832 32536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:35:50.776937 32536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:50.777010 32536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:50.777081 32536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:35:50.777137 32536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:35:50.787170 32536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:33111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:41047,127.18.150.189:40979,127.18.150.188:39071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab
--principal=kudu/127.18.150.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.189:40979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.150.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:50.788458 32536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:50.789718 32536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:50.800640 32541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:50.800714 32544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:50.800719 32542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:50.801775 32536 server_base.cc:1034] running on GCE node
I20260618 22:35:50.802639 32536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:50.804114 32536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:50.805346 32536 hybrid_clock.cc:648] HybridClock initialized: now 1781822150805291 us; error 43 us; skew 500 ppm
I20260618 22:35:50.805688 32536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:35:50.806847 32529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } attempt: 4
I20260618 22:35:50.807869 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.807383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51244)
0618 22:35:50.807653 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.807733 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
Jun 18 22:35:50 dist-test-slave-h7jz krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822150, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:50.808823 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.808498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51258)
0618 22:35:50.808629 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.808713 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260618 22:35:50.808868 32536 init.cc:377] Logged in from keytab as kudu/127.18.150.189@KRBTEST.COM (short username kudu)
W20260618 22:35:50.809036 32529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:40979: Network error: Client connection negotiation failed: client connection to 127.18.150.189:40979: connect: Connection refused (error 111)
I20260618 22:35:50.810417 32536 webserver.cc:460] Webserver started at http://127.18.150.189:45011/ using document root <none> and password file <none>
I20260618 22:35:50.811103 32536 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:50.811218 32536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:50.811511 32536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:35:50.814551 32536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/data/instance:
uuid: "2e0f7d746d014fdd8ab56b168eb48eb2"
format_stamp: "Formatted at 2026-06-18 22:35:50 on dist-test-slave-h7jz"
server_key: "6d300276dff845773d580ffddd65817b"
server_key_iv: "ff43f912e490d1ef31aa001f3158e5c3"
server_key_version: "encryptionkey@0"
I20260618 22:35:50.815325 32536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/wal/instance:
uuid: "2e0f7d746d014fdd8ab56b168eb48eb2"
format_stamp: "Formatted at 2026-06-18 22:35:50 on dist-test-slave-h7jz"
server_key: "6d300276dff845773d580ffddd65817b"
server_key_iv: "ff43f912e490d1ef31aa001f3158e5c3"
server_key_version: "encryptionkey@0"
I20260618 22:35:50.820506 32536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260618 22:35:50.824028 32551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:50.825459 32536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:35:50.825623 32536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/wal
uuid: "2e0f7d746d014fdd8ab56b168eb48eb2"
format_stamp: "Formatted at 2026-06-18 22:35:50 on dist-test-slave-h7jz"
server_key: "6d300276dff845773d580ffddd65817b"
server_key_iv: "ff43f912e490d1ef31aa001f3158e5c3"
server_key_version: "encryptionkey@0"
I20260618 22:35:50.825807 32536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:50.855406 32536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:50.859683 32536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:50.859949 32536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:50.876915 32536 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.189:40979
I20260618 22:35:50.876950 32602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.189:40979 every 8 connection(s)
I20260618 22:35:50.878242 32536 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
I20260618 22:35:50.881918 32603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:50.887477 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 32536
I20260618 22:35:50.887631 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-1/wal/instance
I20260618 22:35:50.888065 19034 external_mini_cluster.cc:1428] Setting key 471a285cf5d26f5d177225d7f74fab51
I20260618 22:35:50.888530 32603 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:35:50 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822150, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.150.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.188@KRBTEST.COM" created.
I20260618 22:35:50.908653 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.891286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41330 (local address 127.18.150.190:41047)
0618 22:35:50.891459 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:50.891465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:50.892822 (+ 1357us) server_negotiation.cc:407] Connection header received
0618 22:35:50.894756 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:50.894762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:50.894848 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:50.894946 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:50.896428 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.897096 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.898647 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.899063 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.899173 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:50.902753 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:50.902758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:50.902761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:50.902797 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:50.905340 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:50.906145 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:50.906151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:50.906154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:50.906308 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:50.906720 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:50.906725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:50.906729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:50.907068 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:50.907211 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:50.908074 (+ 863us) server_negotiation.cc:299] Negotiation successful
0618 22:35:50.908325 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:35:50.909637 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.891827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:41047 (local address 127.0.0.1:41330)
0618 22:35:50.892643 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.892688 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:50.894477 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:50.895173 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:50.895182 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:50.895701 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:50.896254 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:50.896271 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.897241 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.897245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:50.898510 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:50.898522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.899278 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.899284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:50.899501 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:50.900133 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:50.900155 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:50.902507 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:50.905496 (+ 2989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:50.905506 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:50.905525 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:50.906018 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:50.906418 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:50.906423 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:50.906427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:50.906607 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:50.907192 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:50.907201 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:50.907659 (+ 458us) client_negotiation.cc:769] Sending connection context
0618 22:35:50.907970 (+ 311us) client_negotiation.cc:240] Negotiation successful
0618 22:35:50.908498 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":380,"threads_started":1}
I20260618 22:35:50.911643 32603 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:35:50 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822150, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
I20260618 22:35:50.927235 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.912304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51262)
0618 22:35:50.912493 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.912512 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:50.912614 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:50.913803 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:50.913807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:50.914211 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:50.914358 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:50.914366 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.915822 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.915827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:50.917069 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:50.917083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.918022 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.918028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:50.918250 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:50.918852 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:50.918872 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:50.921170 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:50.924909 (+ 3739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:50.924913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:50.924916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:50.925288 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:50.925688 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:50.925691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:50.925693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:50.925808 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:50.926542 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:50.926546 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:50.926661 (+ 115us) client_negotiation.cc:769] Sending connection context
0618 22:35:50.926765 (+ 104us) client_negotiation.cc:240] Negotiation successful
0618 22:35:50.926962 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260618 22:35:50.927592 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.912559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51262 (local address 127.18.150.189:40979)
0618 22:35:50.913288 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:50.913297 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:50.913330 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:35:50.913396 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:50.913404 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:50.913645 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:50.913822 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:50.914517 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.915684 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.917216 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:50.917851 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:50.917999 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:50.921374 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:50.921387 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:50.921405 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:50.921447 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:50.924776 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:50.925420 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:50.925430 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:50.925441 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:50.925579 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:50.925911 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:50.925916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:50.925919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:50.926408 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:50.926634 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:50.926933 (+ 299us) server_negotiation.cc:299] Negotiation successful
0618 22:35:50.927198 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":383,"threads_started":1}
I20260618 22:35:50.929988 32603 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:35:50.931198 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.930704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:60912)
0618 22:35:50.930844 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.931026 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260618 22:35:50.931460 32553 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
I20260618 22:35:50.932909 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.932603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:60924)
0618 22:35:50.932738 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.932812 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260618 22:35:50.933102 32603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:39071: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab.
Entry for principal kudu/127.18.150.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab.
I20260618 22:35:50.937171 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.188:39071
--webserver_interface=127.18.150.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab
--principal=kudu/127.18.150.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:33111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:41047,127.18.150.189:40979,127.18.150.188:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:35:50.984979 32603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } attempt: 1
I20260618 22:35:50.986482 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.985774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:60938)
0618 22:35:50.986041 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.986185 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260618 22:35:50.987532 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:50.987112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:60948)
0618 22:35:50.987288 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:50.987397 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260618 22:35:50.987732 32603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:39071: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
I20260618 22:35:51.056015 32603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } attempt: 2
I20260618 22:35:51.057401 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.056811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:60954)
0618 22:35:51.057065 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.057186 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260618 22:35:51.058487 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.058090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:60962)
0618 22:35:51.058232 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.058324 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260618 22:35:51.058706 32603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:39071: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
I20260618 22:35:51.108311 32529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } attempt: 5
Jun 18 22:35:51 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822150, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
I20260618 22:35:51.122604 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.109127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51274 (local address 127.18.150.189:40979)
0618 22:35:51.109288 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.109292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.109312 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:35:51.109538 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.109541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.109621 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.109702 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.110733 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.111416 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.113076 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.113535 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.113629 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.117451 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.117456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.117460 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.117494 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.120027 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.120809 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.120814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.120846 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.120978 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.121356 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.121359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.121361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.121633 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.121740 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.122194 (+ 454us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.122376 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:35:51.122757 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.108993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51274)
0618 22:35:51.109225 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.109248 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:51.109382 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:51.109771 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:51.109776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:51.110313 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:51.110565 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.110578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.111551 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.111557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.112924 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.112940 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.113698 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.113705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.113909 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.114558 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:51.114582 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:51.117210 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:51.120174 (+ 2964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.120180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.120184 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.120672 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.121089 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.121094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.121097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.121262 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.121739 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:51.121744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:51.121922 (+ 178us) client_negotiation.cc:769] Sending connection context
0618 22:35:51.122113 (+ 191us) client_negotiation.cc:240] Negotiation successful
0618 22:35:51.122425 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260618 22:35:51.123746 32529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:35:51.124853 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.124452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:60974)
0618 22:35:51.124647 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.124726 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260618 22:35:51.125839 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.125534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:60978)
0618 22:35:51.125662 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.125740 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260618 22:35:51.126068 32529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:39071: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
I20260618 22:35:51.149928 32603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } attempt: 3
I20260618 22:35:51.151106 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.150587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:60992)
0618 22:35:51.150809 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.150919 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260618 22:35:51.152191 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.151716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:32772)
0618 22:35:51.151954 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.152051 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260618 22:35:51.152431 32603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:39071: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
I20260618 22:35:51.177258 32529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } attempt: 1
I20260618 22:35:51.178330 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.177857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:32774)
0618 22:35:51.178069 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.178165 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260618 22:35:51.179278 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.178951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:32788)
0618 22:35:51.179098 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.179171 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260618 22:35:51.179466 32529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:39071: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
I20260618 22:35:51.247659 32529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } attempt: 2
I20260618 22:35:51.248816 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.248340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:32804)
0618 22:35:51.248570 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.248655 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260618 22:35:51.249907 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.249518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:32810)
0618 22:35:51.249705 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.249779 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260618 22:35:51.250120 32529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:39071: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
W20260618 22:35:51.259388 32610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:51.259673 32610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:51.259799 32610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:51.266934 32610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:35:51.267050 32610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:51.267125 32610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:51.267205 32610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:35:51.267263 32610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:35:51.277014 32610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:33111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:41047,127.18.150.189:40979,127.18.150.188:39071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab
--principal=kudu/127.18.150.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.188:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.150.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:51.278270 32610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:51.279502 32610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:51.290606 32618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:51.290572 32616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:51.290640 32615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:51.291006 32610 server_base.cc:1034] running on GCE node
I20260618 22:35:51.291795 32610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:51.293099 32610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:51.294292 32610 hybrid_clock.cc:648] HybridClock initialized: now 1781822151294239 us; error 50 us; skew 500 ppm
I20260618 22:35:51.294620 32610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:35:51.295616 32603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } attempt: 4
I20260618 22:35:51.296698 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.296136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:32818)
0618 22:35:51.296400 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.296506 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
Jun 18 22:35:51 dist-test-slave-h7jz krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822151, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:51.297729 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.297387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:32832)
0618 22:35:51.297556 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.297626 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260618 22:35:51.297930 32603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:39071: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
I20260618 22:35:51.298132 32610 init.cc:377] Logged in from keytab as kudu/127.18.150.188@KRBTEST.COM (short username kudu)
I20260618 22:35:51.299672 32610 webserver.cc:460] Webserver started at http://127.18.150.188:45557/ using document root <none> and password file <none>
I20260618 22:35:51.300369 32610 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:51.300493 32610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:51.300796 32610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:35:51.303845 32610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/data/instance:
uuid: "51a5a876cad142ca953fbbcc4112fb3e"
format_stamp: "Formatted at 2026-06-18 22:35:51 on dist-test-slave-h7jz"
server_key: "5ad7fc6bf24bc0382cee9ab010b6ddd2"
server_key_iv: "234b5b0d2a12117dc9f5645775f028cd"
server_key_version: "encryptionkey@0"
I20260618 22:35:51.304659 32610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/wal/instance:
uuid: "51a5a876cad142ca953fbbcc4112fb3e"
format_stamp: "Formatted at 2026-06-18 22:35:51 on dist-test-slave-h7jz"
server_key: "5ad7fc6bf24bc0382cee9ab010b6ddd2"
server_key_iv: "234b5b0d2a12117dc9f5645775f028cd"
server_key_version: "encryptionkey@0"
I20260618 22:35:51.310551 32610 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260618 22:35:51.314200 32625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:51.315536 32610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:35:51.315686 32610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/wal
uuid: "51a5a876cad142ca953fbbcc4112fb3e"
format_stamp: "Formatted at 2026-06-18 22:35:51 on dist-test-slave-h7jz"
server_key: "5ad7fc6bf24bc0382cee9ab010b6ddd2"
server_key_iv: "234b5b0d2a12117dc9f5645775f028cd"
server_key_version: "encryptionkey@0"
I20260618 22:35:51.315869 32610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:51.337177 32610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:51.341182 32610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:51.341428 32610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:51.356308 32529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } attempt: 3
I20260618 22:35:51.357479 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.356970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:32838)
0618 22:35:51.357196 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.357307 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:39071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260618 22:35:51.357654 32610 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.188:39071
I20260618 22:35:51.357672 32676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.188:39071 every 8 connection(s)
I20260618 22:35:51.359138 32610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
I20260618 22:35:51.361256 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 32610
I20260618 22:35:51.361452 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821960891835-19034-0/minicluster-data/master-2/wal/instance
I20260618 22:35:51.362061 19034 external_mini_cluster.cc:1428] Setting key 70fdd641d861ea1206c4b09a3a9cf7f8
I20260618 22:35:51.363646 32678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:51.371315 32678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:35:51 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822150, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
Jun 18 22:35:51 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
I20260618 22:35:51.381772 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.358199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:32850)
0618 22:35:51.358375 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.358391 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:51.358514 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:51.363336 (+ 4822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:51.363343 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:51.363909 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:51.364137 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.364149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.368970 (+ 4821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.368976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.370450 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.370468 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.371675 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.371681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.371861 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.372417 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:51.372444 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:51.375106 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:51.378884 (+ 3778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.378890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.378894 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.379383 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.379821 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.379825 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.379828 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.379960 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.380855 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:51.380859 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:51.381027 (+ 168us) client_negotiation.cc:769] Sending connection context
0618 22:35:51.381146 (+ 119us) client_negotiation.cc:240] Negotiation successful
0618 22:35:51.381420 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
Jun 18 22:35:51 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822151, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:51.383461 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.359121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.18.150.188:39071)
0618 22:35:51.359890 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.359902 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.359963 (+ 61us) server_negotiation.cc:407] Connection header received
0618 22:35:51.361842 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.361886 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.362361 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.362727 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.366525 (+ 3798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.368767 (+ 2242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.370605 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.371521 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.371691 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.375413 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.375423 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.375440 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.375484 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.378733 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.379518 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.379526 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.379535 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.379700 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.380059 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.380064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.380068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.380736 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.380903 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.381406 (+ 503us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.381838 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"spinlock_wait_cycles":19328,"thread_start_us":299,"threads_started":1}
I20260618 22:35:51.386622 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.363017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.18.150.188:39071)
0618 22:35:51.369351 (+ 6334us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.369355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.369371 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:35:51.369406 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.369409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.369466 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.369554 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.371215 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.372487 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.374322 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.375052 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.375153 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.379596 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.379602 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.379607 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.379643 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.383605 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.384528 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.384544 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.384548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.384690 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.385149 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.385154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.385158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.385487 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.385759 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.386114 (+ 355us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.386307 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6223,"thread_start_us":6074,"threads_started":1}
I20260618 22:35:51.390015 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.374009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:41047 (local address 127.0.0.1:41332)
0618 22:35:51.374641 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.374685 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:51.374878 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:51.375470 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:51.375477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:51.376210 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:51.376615 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.376632 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.378002 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.378007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.379359 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.379370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.380313 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.380319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.380504 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.381440 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:51.381457 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:51.383673 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:51.387177 (+ 3504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.387186 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.387193 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.387682 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.388040 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.388045 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.388050 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.388176 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.388956 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:51.388963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:51.389138 (+ 175us) client_negotiation.cc:769] Sending connection context
0618 22:35:51.389316 (+ 178us) client_negotiation.cc:240] Negotiation successful
0618 22:35:51.389610 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":269,"threads_started":1}
I20260618 22:35:51.390035 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.373996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41332 (local address 127.18.150.190:41047)
0618 22:35:51.374159 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.374163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.374767 (+ 604us) server_negotiation.cc:407] Connection header received
0618 22:35:51.375091 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.375096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.375162 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.375238 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.376762 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.377782 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.379493 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.380120 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.380241 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.383883 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.383888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.383891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.383921 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.387064 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.387797 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.387804 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.387807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.387943 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.388308 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.388317 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.388320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.388706 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.388849 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.389475 (+ 626us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.389666 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:35:51.390935 32529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59: Bootstrap starting.
I20260618 22:35:51.391453 32678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:35:51.394766 32529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59: Neither blocks nor log segments found. Creating new log.
I20260618 22:35:51.395792 32529 log.cc:824] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:35:51 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:51.399194 32529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59: No bootstrap required, opened a new log
Jun 18 22:35:51 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822151, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
I20260618 22:35:51.406239 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.388446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41346 (local address 127.18.150.190:41047)
0618 22:35:51.389029 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.389034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.389054 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:35:51.389096 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.389101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.389167 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.390893 (+ 1726us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.391162 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.392313 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.393865 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.394349 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.394464 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.399275 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.399282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.399288 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.399326 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.402238 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.402948 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.402954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.402958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.403093 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.403456 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.403460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.403462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.403736 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.405660 (+ 1924us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.405763 (+ 103us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.405930 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":265,"threads_started":1}
I20260618 22:35:51.407114 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.392403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51278 (local address 127.18.150.189:40979)
0618 22:35:51.392588 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.392594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.392614 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:35:51.392674 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.392679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.392753 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.392856 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.393669 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.394675 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.396611 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.397209 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.397322 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.400445 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.400452 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.400456 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.400497 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.404004 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.404797 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.404802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.404806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.404978 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.405407 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.405412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.405416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.405734 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.405856 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.406535 (+ 679us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.406750 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260618 22:35:51.407656 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.392059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:40979 (local address 127.0.0.1:51278)
0618 22:35:51.392224 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.392244 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:51.392338 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:51.392898 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:51.392903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:51.393331 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:51.393503 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.393514 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.394827 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.394832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.396389 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.396405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.397358 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.397363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.397582 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.398155 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:51.398179 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:51.400248 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:51.404156 (+ 3908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.404162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.404166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.404661 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.405125 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.405130 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.405133 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.405291 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.405993 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:51.405998 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:51.406150 (+ 152us) client_negotiation.cc:769] Sending connection context
0618 22:35:51.407066 (+ 916us) client_negotiation.cc:240] Negotiation successful
0618 22:35:51.407335 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260618 22:35:51.408736 32678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:35:51.410686 32529 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } }
I20260618 22:35:51.411157 32529 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:51.411322 32529 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2da061c09aa45e88de430ab923e5e59, State: Initialized, Role: FOLLOWER
I20260618 22:35:51.412329 32529 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } }
Jun 18 22:35:51 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822151, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
Jun 18 22:35:51 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
I20260618 22:35:51.423092 32685 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } } }
I20260618 22:35:51.423506 32685 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:35:51.424346 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.409480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:32870)
0618 22:35:51.409658 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.409681 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:51.409783 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:51.410372 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:51.410376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:51.410811 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:51.410982 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.410993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.412242 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.412246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.413547 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.413560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.414508 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.414513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.414687 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.415139 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:51.415160 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:51.417649 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:51.421653 (+ 4004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.421658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.421662 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.422145 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.422527 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.422531 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.422534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.422675 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.423342 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:51.423347 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:51.423501 (+ 154us) client_negotiation.cc:769] Sending connection context
0618 22:35:51.423683 (+ 182us) client_negotiation.cc:240] Negotiation successful
0618 22:35:51.423995 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260618 22:35:51.424520 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.409849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.18.150.188:39071)
0618 22:35:51.410013 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.410019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.410033 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:35:51.410085 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.410089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.410152 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.410233 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.411112 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.412103 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.413666 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.414251 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.414324 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.417823 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.417829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.417834 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.417867 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.421509 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.422256 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.422261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.422264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.422391 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.422791 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.422795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.422798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.423165 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.423289 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.423806 (+ 517us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.424164 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":31}
I20260618 22:35:51.425083 32529 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:35:51.425407 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.408095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.18.150.189:40979)
0618 22:35:51.408265 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.408271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.408289 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:35:51.408353 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.408358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.408426 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.408509 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.410096 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.411111 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.412799 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.413447 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.413558 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.418747 (+ 5189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.418754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.418759 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.418792 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.421796 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.422691 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.422696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.422699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.422854 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.423292 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.423297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.423300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.423617 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.423866 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.424901 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.425117 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:35:51.430450 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:35:51.431603 32678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e: Bootstrap starting.
I20260618 22:35:51.435089 32678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e: Neither blocks nor log segments found. Creating new log.
W20260618 22:35:51.435695 32696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:35:51.435879 32696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:35:51.436384 32678 log.cc:824] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:51.448801 32678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e: No bootstrap required, opened a new log
I20260618 22:35:51.438149 32462 master_runner.cc:186] Error getting master registration for 127.18.150.189:40979: OK, instance_id { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" instance_seqno: 1781822150871900 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:35:51.449795 32462 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:35:51.460080 32678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } }
I20260618 22:35:51.460572 32678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:51.460698 32678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a5a876cad142ca953fbbcc4112fb3e, State: Initialized, Role: FOLLOWER
I20260618 22:35:51.461489 32678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } }
I20260618 22:35:51.468467 32705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } } }
I20260618 22:35:51.468917 32705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:35:51.470865 32678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:35:51.483454 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.451743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32878 (local address 127.18.150.188:39071)
0618 22:35:51.451924 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.451930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.465368 (+ 13438us) server_negotiation.cc:407] Connection header received
0618 22:35:51.465519 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.465525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.465610 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.465704 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.468199 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.469397 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.471564 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.472337 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.472435 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.476591 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.476599 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.476604 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.476648 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.480231 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.481122 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.481127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.481131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.481267 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.481700 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.481705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.481708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.482113 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.482325 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.482866 (+ 541us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.483094 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:35:51.486939 32685 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:35:51.487264 32685 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } }
I20260618 22:35:51.489042 32685 leader_election.cc:288] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e0f7d746d014fdd8ab56b168eb48eb2 (127.18.150.189:40979), 51a5a876cad142ca953fbbcc4112fb3e (127.18.150.188:39071)
I20260618 22:35:51.490914 32652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a2da061c09aa45e88de430ab923e5e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a5a876cad142ca953fbbcc4112fb3e" is_pre_election: true
I20260618 22:35:51.491528 32652 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2da061c09aa45e88de430ab923e5e59 in term 0.
I20260618 22:35:51.492220 32479 leader_election.cc:302] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51a5a876cad142ca953fbbcc4112fb3e, a2da061c09aa45e88de430ab923e5e59; no voters:
I20260618 22:35:51.492708 32685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:35:51.492864 32685 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:35:51.492983 32685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:51.494279 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.461358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51300 (local address 127.18.150.189:40979)
0618 22:35:51.461570 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.461576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.478524 (+ 16948us) server_negotiation.cc:407] Connection header received
0618 22:35:51.478777 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.478783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.478873 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.478973 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.480768 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.481817 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.483993 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.484611 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.484704 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.487663 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.487669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.487674 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.487724 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.491015 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.491930 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.491936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.491939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.492088 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.492532 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.492537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.492541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.492920 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.493139 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.493705 (+ 566us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.493928 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260618 22:35:51.495011 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.451871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41354 (local address 127.18.150.190:41047)
0618 22:35:51.452074 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.452080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.462442 (+ 10362us) server_negotiation.cc:407] Connection header received
0618 22:35:51.462651 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.462657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.462787 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.462917 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.478743 (+ 15826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.479865 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.482079 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.482774 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.482871 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.486874 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.486881 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.486885 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.486931 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.490743 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.491931 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.491951 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.491953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.492088 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.492600 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.492605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.492608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.493134 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.493366 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.494222 (+ 856us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.494652 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:35:51.496295 32685 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } }
I20260618 22:35:51.497605 32685 leader_election.cc:288] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [CANDIDATE]: Term 1 election: Requested vote from peers 2e0f7d746d014fdd8ab56b168eb48eb2 (127.18.150.189:40979), 51a5a876cad142ca953fbbcc4112fb3e (127.18.150.188:39071)
I20260618 22:35:51.497905 32652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a2da061c09aa45e88de430ab923e5e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a5a876cad142ca953fbbcc4112fb3e"
I20260618 22:35:51.498136 32652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:51.501264 32652 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2da061c09aa45e88de430ab923e5e59 in term 1.
I20260618 22:35:51.501645 32479 leader_election.cc:302] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51a5a876cad142ca953fbbcc4112fb3e, a2da061c09aa45e88de430ab923e5e59; no voters:
I20260618 22:35:51.501986 32685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:35:51.514211 32685 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [term 1 LEADER]: Becoming Leader. State: Replica: a2da061c09aa45e88de430ab923e5e59, State: Running, Role: LEADER
I20260618 22:35:51.514885 32685 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } }
I20260618 22:35:51.518656 32710 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2da061c09aa45e88de430ab923e5e59. Latest consensus state: current_term: 1 leader_uuid: "a2da061c09aa45e88de430ab923e5e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } } }
I20260618 22:35:51.519016 32710 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:51.526417 32714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:35:51.528584 32714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:35:51.539364 32652 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [term 1 FOLLOWER]: Refusing update from remote peer a2da061c09aa45e88de430ab923e5e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:35:51.540246 32710 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:35:51.549038 32705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2da061c09aa45e88de430ab923e5e59. Latest consensus state: current_term: 1 leader_uuid: "a2da061c09aa45e88de430ab923e5e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } } }
I20260618 22:35:51.549360 32705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:35:51.553736 32717 mvcc.cc:205] Tried to move back new op lower bound from 7298343532694990848 to 7298343532609953792. Current Snapshot: MvccSnapshot[applied={T|T < 7298343532694990848}]
I20260618 22:35:51.554251 32685 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2da061c09aa45e88de430ab923e5e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } } }
I20260618 22:35:51.554605 32685 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:51.557327 32720 mvcc.cc:205] Tried to move back new op lower bound from 7298343532694990848 to 7298343532609953792. Current Snapshot: MvccSnapshot[applied={T|T < 7298343532694990848}]
I20260618 22:35:51.558230 32714 catalog_manager.cc:1347] Generated new cluster ID: 1fb63cbbe1b241b799acc1b55b69f841
I20260618 22:35:51.558326 32714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:35:51.563176 32705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2da061c09aa45e88de430ab923e5e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } } }
I20260618 22:35:51.563441 32705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:35:51.590946 32725 catalog_manager.cc:1259] Loaded cluster ID: 1fb63cbbe1b241b799acc1b55b69f841
I20260618 22:35:51.591048 32725 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e: loading cluster ID for follower catalog manager: success
W20260618 22:35:51.592272 32725 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260618 22:35:51.592422 32725 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:35:51.597154 32603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } attempt: 5
I20260618 22:35:51.601763 32714 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:35:51.606190 32714 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:35:51.591547 32610 master_runner.cc:186] Error getting master registration for 127.18.150.189:40979: OK, instance_id { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" instance_seqno: 1781822150871900 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:35:51.607985 32610 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 18 22:35:51 dist-test-slave-h7jz krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822150, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
I20260618 22:35:51.620795 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.598497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32882 (local address 127.18.150.188:39071)
0618 22:35:51.598666 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:51.598671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:51.598692 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:35:51.598743 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:51.598748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:51.598822 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:51.598915 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:51.600988 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.602356 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.604060 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.604832 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.604951 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.609007 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:51.609014 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:51.609019 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:51.609065 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:51.612658 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.613515 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.613521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.613524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.613679 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:51.618136 (+ 4457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:51.618142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:51.618145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:51.618561 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:51.618737 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:51.619405 (+ 668us) server_negotiation.cc:299] Negotiation successful
0618 22:35:51.620459 (+ 1054us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:35:51.620275 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:51.597773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:39071 (local address 127.0.0.1:32882)
0618 22:35:51.598023 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:51.598044 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:51.598167 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:51.599117 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:51.599122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:51.599640 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:51.599847 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.599860 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.602495 (+ 2635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.602500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.603879 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:51.603896 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:51.605151 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:51.605157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:51.605395 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:51.606085 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:51.606108 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:51.608769 (+ 2661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:51.612866 (+ 4097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.612872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.612875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.613348 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.613813 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:51.613818 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:51.613822 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:51.617992 (+ 4170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:51.618804 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:51.618810 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:51.618979 (+ 169us) client_negotiation.cc:769] Sending connection context
0618 22:35:51.619119 (+ 140us) client_negotiation.cc:240] Negotiation successful
0618 22:35:51.619865 (+ 746us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260618 22:35:51.627578 32603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2: Bootstrap starting.
I20260618 22:35:51.631938 32603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2: Neither blocks nor log segments found. Creating new log.
I20260618 22:35:51.633390 32603 log.cc:824] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:51.649286 32714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59: Generated new TSK 0
I20260618 22:35:51.651795 32714 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:35:51.667472 32603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2: No bootstrap required, opened a new log
I20260618 22:35:51.692114 32603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } }
I20260618 22:35:51.692890 32603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:51.693064 32603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e0f7d746d014fdd8ab56b168eb48eb2, State: Initialized, Role: FOLLOWER
I20260618 22:35:51.693874 32603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } }
I20260618 22:35:51.714509 32731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } } }
I20260618 22:35:51.714952 32731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:35:51.716150 32603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:35:51.716223 32577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a2da061c09aa45e88de430ab923e5e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2"
I20260618 22:35:51.716737 32577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:51.719461 32578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a2da061c09aa45e88de430ab923e5e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" is_pre_election: true
I20260618 22:35:51.721774 32577 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2da061c09aa45e88de430ab923e5e59 in term 1.
I20260618 22:35:51.722498 32576 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [term 1 FOLLOWER]: Refusing update from remote peer a2da061c09aa45e88de430ab923e5e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:35:51.724215 32710 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:35:51.741621 32731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2da061c09aa45e88de430ab923e5e59. Latest consensus state: current_term: 1 leader_uuid: "a2da061c09aa45e88de430ab923e5e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } } }
I20260618 22:35:51.741991 32731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:35:51.742321 32736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2da061c09aa45e88de430ab923e5e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } } }
I20260618 22:35:51.742671 32736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:35:51.756683 32733 mvcc.cc:205] Tried to move back new op lower bound from 7298343533134848000 to 7298343532609953792. Current Snapshot: MvccSnapshot[applied={T|T < 7298343533134848000 or (T in {7298343533134848000})}]
I20260618 22:35:51.796558 32747 catalog_manager.cc:1259] Loaded cluster ID: 1fb63cbbe1b241b799acc1b55b69f841
I20260618 22:35:51.797104 32747 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2: loading cluster ID for follower catalog manager: success
I20260618 22:35:51.801103 32747 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2: acquiring CA information for follower catalog manager: success
I20260618 22:35:51.803742 32747 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2e0f7d746d014fdd8ab56b168eb48eb2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:35:52.163661 32710 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2da061c09aa45e88de430ab923e5e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2da061c09aa45e88de430ab923e5e59" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41047 } } peers { permanent_uuid: "2e0f7d746d014fdd8ab56b168eb48eb2" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 40979 } } peers { permanent_uuid: "51a5a876cad142ca953fbbcc4112fb3e" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 39071 } } }
I20260618 22:35:52.164005 32710 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2da061c09aa45e88de430ab923e5e59 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:52.595633 32725 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e: acquiring CA information for follower catalog manager: success
I20260618 22:35:52.597602 32725 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 51a5a876cad142ca953fbbcc4112fb3e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:35:52.772037 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 32462
I20260618 22:35:52.773166 32524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:52.885042 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 32462
W20260618 22:35:52.901353 32555 connection.cc:535] server connection from 127.0.0.1:51274 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 22:35:52.901698 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 32536
I20260618 22:35:52.902707 32598 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:53.007540 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 32536
I20260618 22:35:53.023514 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 32610
I20260618 22:35:53.024811 32672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:53.129254 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 32610
2026-06-18T22:35:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4344 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:35:53 dist-test-slave-h7jz krb5kdc[32757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:35:53 dist-test-slave-h7jz krb5kdc[32757](info): set up 2 sockets
Jun 18 22:35:53 dist-test-slave-h7jz krb5kdc[32757](info): commencing operation
krb5kdc: starting...
W20260618 22:35:54.142107 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.008s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:35:54 dist-test-slave-h7jz krb5kdc[32757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:35:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:35:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:35:54.357515 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38089
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:38089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:35:54.672972 302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:54.673285 302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:54.673434 302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:54.680685 302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:35:54.680799 302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:54.680876 302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:54.680953 302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:35:54.681012 302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:35:54.690778 302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:38089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:54.692075 302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:54.693333 302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:54.704523 307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:54.704604 308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:54.704633 310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:54.705322 302 server_base.cc:1034] running on GCE node
I20260618 22:35:54.706120 302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:54.707453 302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:54.708663 302 hybrid_clock.cc:648] HybridClock initialized: now 1781822154708602 us; error 45 us; skew 500 ppm
I20260618 22:35:54.708976 302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:54 dist-test-slave-h7jz krb5kdc[32757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822154, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:54.711839 302 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:35:54.713177 302 webserver.cc:460] Webserver started at http://127.18.150.190:46791/ using document root <none> and password file <none>
I20260618 22:35:54.713832 302 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:54.713966 302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:54.714258 302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:35:54.717263 302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "7d34f43c8835469797f84eae1e290420"
format_stamp: "Formatted at 2026-06-18 22:35:54 on dist-test-slave-h7jz"
server_key: "05f0180f31a7155a184211edced14684"
server_key_iv: "e8080b2af8d97705c20ebbc1b6723136"
server_key_version: "encryptionkey@0"
I20260618 22:35:54.718071 302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "7d34f43c8835469797f84eae1e290420"
format_stamp: "Formatted at 2026-06-18 22:35:54 on dist-test-slave-h7jz"
server_key: "05f0180f31a7155a184211edced14684"
server_key_iv: "e8080b2af8d97705c20ebbc1b6723136"
server_key_version: "encryptionkey@0"
I20260618 22:35:54.722817 302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:35:54.726017 317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:54.727263 302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:35:54.727411 302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "7d34f43c8835469797f84eae1e290420"
format_stamp: "Formatted at 2026-06-18 22:35:54 on dist-test-slave-h7jz"
server_key: "05f0180f31a7155a184211edced14684"
server_key_iv: "e8080b2af8d97705c20ebbc1b6723136"
server_key_version: "encryptionkey@0"
I20260618 22:35:54.727591 302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:54.749034 302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:54.753037 302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:54.753311 302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:54.770308 302 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:38089
I20260618 22:35:54.770325 368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:38089 every 8 connection(s)
I20260618 22:35:54.771621 302 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:35:54.775223 369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:54.777575 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 302
I20260618 22:35:54.777761 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:35:54.778431 19034 external_mini_cluster.cc:1428] Setting key 2fda32251b8d3f7032683bc7e4fb6cae
I20260618 22:35:54.786494 369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420: Bootstrap starting.
I20260618 22:35:54.790004 369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420: Neither blocks nor log segments found. Creating new log.
I20260618 22:35:54.791234 369 log.cc:824] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:35:54 dist-test-slave-h7jz krb5kdc[32757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:54.794183 369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420: No bootstrap required, opened a new log
I20260618 22:35:54.802357 372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:54.780260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56398 (local address 127.18.150.190:38089)
0618 22:35:54.780988 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:54.781000 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:54.781038 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:35:54.782714 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:54.782737 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:54.783132 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:54.783450 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:54.785119 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:54.786393 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:54.788237 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:54.788892 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:54.789032 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:54.793468 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:54.793479 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:54.793497 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:54.793545 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:54.797538 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:54.798300 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:54.798309 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:54.798317 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:54.798553 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:54.798985 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:54.798989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:54.798993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:54.799751 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:54.800057 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:54.800430 (+ 373us) server_negotiation.cc:299] Negotiation successful
0618 22:35:54.800930 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":328,"threads_started":1}
I20260618 22:35:54.804466 369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d34f43c8835469797f84eae1e290420" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38089 } }
I20260618 22:35:54.804899 369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:54.805007 369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d34f43c8835469797f84eae1e290420, State: Initialized, Role: FOLLOWER
I20260618 22:35:54.805610 369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d34f43c8835469797f84eae1e290420" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38089 } }
I20260618 22:35:54.805852 369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:35:54.806001 369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:35:54.806176 369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:54.806320 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:35:54.808308 369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d34f43c8835469797f84eae1e290420" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38089 } }
I20260618 22:35:54.808830 369 leader_election.cc:302] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d34f43c8835469797f84eae1e290420; no voters:
I20260618 22:35:54.809432 369 leader_election.cc:288] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:35:54.809535 374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:35:54.810186 374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [term 1 LEADER]: Becoming Leader. State: Replica: 7d34f43c8835469797f84eae1e290420, State: Running, Role: LEADER
I20260618 22:35:54.810686 374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d34f43c8835469797f84eae1e290420" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38089 } }
I20260618 22:35:54.811412 369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:35:54.813311 375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d34f43c8835469797f84eae1e290420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d34f43c8835469797f84eae1e290420" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38089 } } }
I20260618 22:35:54.813732 375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:54.814164 376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d34f43c8835469797f84eae1e290420. Latest consensus state: current_term: 1 leader_uuid: "7d34f43c8835469797f84eae1e290420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d34f43c8835469797f84eae1e290420" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38089 } } }
I20260618 22:35:54.814445 376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:54.815421 383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:35:54.820580 383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:35:54.828418 383 catalog_manager.cc:1347] Generated new cluster ID: c3a5a01164b946298a463c43d657ad9b
I20260618 22:35:54.828526 383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:35:54.836627 383 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:35:54.838498 383 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:35:54.848111 383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d34f43c8835469797f84eae1e290420: Generated new TSK 0
I20260618 22:35:54.849149 383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:35:54.866034 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 302
I20260618 22:35:54.867128 364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:54.955865 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 302
2026-06-18T22:35:54Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1822 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:35:55 dist-test-slave-h7jz krb5kdc[397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:35:55 dist-test-slave-h7jz krb5kdc[397](info): set up 2 sockets
Jun 18 22:35:55 dist-test-slave-h7jz krb5kdc[397](info): commencing operation
krb5kdc: starting...
W20260618 22:35:55.946172 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.958s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:35:56 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:35:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:35:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:35:56.158248 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39461
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:39461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:35:56.478283 410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:56.478574 410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:56.478677 410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:56.486125 410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:35:56.486243 410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:56.486301 410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:56.486363 410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:35:56.486413 410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:35:56.496290 410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:39461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:56.497572 410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:56.498872 410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:56.510504 415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:56.510463 418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:56.510532 416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:56.511229 410 server_base.cc:1034] running on GCE node
I20260618 22:35:56.511988 410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:56.513367 410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:56.514566 410 hybrid_clock.cc:648] HybridClock initialized: now 1781822156514508 us; error 50 us; skew 500 ppm
I20260618 22:35:56.514884 410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:56 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822156, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:56.517833 410 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:35:56.519343 410 webserver.cc:460] Webserver started at http://127.18.150.190:42805/ using document root <none> and password file <none>
I20260618 22:35:56.520004 410 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:56.520115 410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:56.520387 410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:35:56.523319 410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "c7c524f35d3d4eca8221fd29b792a6b9"
format_stamp: "Formatted at 2026-06-18 22:35:56 on dist-test-slave-h7jz"
server_key: "b70daf49baa51442803614b38d2184cd"
server_key_iv: "60a16dd8d86cc54c00a320846d0f9cc1"
server_key_version: "encryptionkey@0"
I20260618 22:35:56.524031 410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "c7c524f35d3d4eca8221fd29b792a6b9"
format_stamp: "Formatted at 2026-06-18 22:35:56 on dist-test-slave-h7jz"
server_key: "b70daf49baa51442803614b38d2184cd"
server_key_iv: "60a16dd8d86cc54c00a320846d0f9cc1"
server_key_version: "encryptionkey@0"
I20260618 22:35:56.528909 410 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260618 22:35:56.532188 425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:56.533506 410 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:35:56.533667 410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "c7c524f35d3d4eca8221fd29b792a6b9"
format_stamp: "Formatted at 2026-06-18 22:35:56 on dist-test-slave-h7jz"
server_key: "b70daf49baa51442803614b38d2184cd"
server_key_iv: "60a16dd8d86cc54c00a320846d0f9cc1"
server_key_version: "encryptionkey@0"
I20260618 22:35:56.533846 410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:56.546825 410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:56.550930 410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:56.551182 410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:56.567523 410 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:39461
I20260618 22:35:56.567550 476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:39461 every 8 connection(s)
I20260618 22:35:56.568833 410 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:35:56.572359 477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:56.577809 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 410
I20260618 22:35:56.578047 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:35:56.578677 19034 external_mini_cluster.cc:1428] Setting key 9d278563908f3e68aa1c3e99a70baee7
I20260618 22:35:56.582660 477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Bootstrap starting.
I20260618 22:35:56.586128 477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Neither blocks nor log segments found. Creating new log.
I20260618 22:35:56.587163 477 log.cc:824] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:56.590108 477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: No bootstrap required, opened a new log
Jun 18 22:35:56 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:56.598049 477 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:35:56.598407 477 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:56.598511 477 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7c524f35d3d4eca8221fd29b792a6b9, State: Initialized, Role: FOLLOWER
I20260618 22:35:56.599068 477 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:35:56.599289 477 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:35:56.599436 477 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:35:56.599614 477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:56.600958 480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:56.580543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33960 (local address 127.18.150.190:39461)
0618 22:35:56.581262 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:56.581273 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:56.581312 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:35:56.582977 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:56.582999 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:56.583374 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:56.583694 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:56.585446 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:56.586804 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:56.588436 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:56.589087 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:56.589213 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:56.593678 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:56.593689 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:56.593703 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:56.593740 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:56.596720 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:56.597394 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:56.597401 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:56.597407 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:56.597569 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:56.597959 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:56.597964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:56.597967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:56.598560 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:56.598818 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:56.599487 (+ 669us) server_negotiation.cc:299] Negotiation successful
0618 22:35:56.599908 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":324,"threads_started":1}
I20260618 22:35:56.602393 477 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:35:56.602892 477 leader_election.cc:302] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7c524f35d3d4eca8221fd29b792a6b9; no voters:
I20260618 22:35:56.603458 477 leader_election.cc:288] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:35:56.603636 482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:35:56.604714 482 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 1 LEADER]: Becoming Leader. State: Replica: c7c524f35d3d4eca8221fd29b792a6b9, State: Running, Role: LEADER
I20260618 22:35:56.605261 482 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:35:56.605903 477 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:35:56.607692 483 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } } }
I20260618 22:35:56.608637 483 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:56.609989 484 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7c524f35d3d4eca8221fd29b792a6b9. Latest consensus state: current_term: 1 leader_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } } }
I20260618 22:35:56.610376 484 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:56.614741 491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:35:56.618851 491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:35:56.627029 491 catalog_manager.cc:1347] Generated new cluster ID: 6cda1eaba9f64238a81e7dec29335c02
I20260618 22:35:56.627141 491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:35:56.642383 491 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:35:56.644140 491 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:35:56.653478 491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Generated new TSK 0
I20260618 22:35:56.654525 491 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:35:56.703502 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39461
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:35:57.027566 503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:57.027840 503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:57.027936 503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:57.035326 503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:57.035451 503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:57.035580 503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:35:57.045434 503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:57.046663 503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:57.047943 503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:57.059901 508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:57.060039 509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:57.061110 511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:57.061244 503 server_base.cc:1034] running on GCE node
I20260618 22:35:57.061844 503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:57.062868 503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:57.064066 503 hybrid_clock.cc:648] HybridClock initialized: now 1781822157064010 us; error 50 us; skew 500 ppm
I20260618 22:35:57.064357 503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:57 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822157, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:57.067276 503 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:35:57.068895 503 webserver.cc:460] Webserver started at http://127.18.150.129:43483/ using document root <none> and password file <none>
I20260618 22:35:57.069552 503 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:57.069665 503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:57.069967 503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:35:57.072733 503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "ceeffefe2d6f4430971feaeef9474273"
format_stamp: "Formatted at 2026-06-18 22:35:57 on dist-test-slave-h7jz"
server_key: "8a745f92ca8f614560a142606547c688"
server_key_iv: "a11056c022482671461af1a918768374"
server_key_version: "encryptionkey@0"
I20260618 22:35:57.073444 503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "ceeffefe2d6f4430971feaeef9474273"
format_stamp: "Formatted at 2026-06-18 22:35:57 on dist-test-slave-h7jz"
server_key: "8a745f92ca8f614560a142606547c688"
server_key_iv: "a11056c022482671461af1a918768374"
server_key_version: "encryptionkey@0"
I20260618 22:35:57.080334 503 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260618 22:35:57.083642 518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:57.084980 503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:35:57.085148 503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "ceeffefe2d6f4430971feaeef9474273"
format_stamp: "Formatted at 2026-06-18 22:35:57 on dist-test-slave-h7jz"
server_key: "8a745f92ca8f614560a142606547c688"
server_key_iv: "a11056c022482671461af1a918768374"
server_key_version: "encryptionkey@0"
I20260618 22:35:57.085319 503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:57.103720 503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:57.107762 503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:57.108014 503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:57.108963 503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:35:57.110329 503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:35:57.110406 503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:57.110522 503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:35:57.110571 503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:57.139426 503 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:43581
I20260618 22:35:57.139469 630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:43581 every 8 connection(s)
I20260618 22:35:57.140630 503 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:35:57.144064 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 503
I20260618 22:35:57.144248 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:35:57.144925 19034 external_mini_cluster.cc:1428] Setting key a05e75b8e0a54b6f4a8b684a4f6deca2
Jun 18 22:35:57 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822157, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:57.160320 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:57.143058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:39155 (local address 127.18.150.190:39461)
0618 22:35:57.143487 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:57.143492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:57.144523 (+ 1031us) server_negotiation.cc:407] Connection header received
0618 22:35:57.146510 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:57.146514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:57.146594 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:57.146698 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:57.148171 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:57.149174 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:57.150706 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:57.151271 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:57.151391 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:57.155018 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:57.155023 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:57.155026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:57.155057 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:57.157464 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:57.158277 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:57.158281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:57.158283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:57.158395 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:57.158810 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:57.158813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:57.158815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:57.159076 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:57.159181 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:57.159898 (+ 717us) server_negotiation.cc:299] Negotiation successful
0618 22:35:57.160070 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":214,"threads_started":1}
I20260618 22:35:57.161604 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:57.143618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:39155)
0618 22:35:57.144362 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:57.144411 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:57.146232 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:57.146952 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:57.146963 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:57.147451 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:57.148002 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:57.148022 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:57.149295 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:57.149298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:57.150578 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:57.150590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:57.151446 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:57.151450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:57.151673 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:57.152273 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:57.152298 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:57.154810 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:57.157589 (+ 2779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:57.157597 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:57.157612 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:57.158171 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:57.158492 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:57.158496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:57.158500 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:57.158716 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:57.159180 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:57.159187 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:57.159556 (+ 369us) client_negotiation.cc:769] Sending connection context
0618 22:35:57.159838 (+ 282us) client_negotiation.cc:240] Negotiation successful
0618 22:35:57.160318 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":307,"threads_started":1}
I20260618 22:35:57.163108 631 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39461
I20260618 22:35:57.163420 631 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:57.164304 631 heartbeater.cc:511] Master 127.18.150.190:39461 requested a full tablet report, sending...
I20260618 22:35:57.166596 442 ts_manager.cc:194] Registered new tserver with Master: ceeffefe2d6f4430971feaeef9474273 (127.18.150.129:43581)
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
I20260618 22:35:57.168069 442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:39155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:35:57.192054 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39461
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:35:57.498421 637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:57.498692 637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:57.498821 637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:57.505729 637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:57.505836 637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:57.505998 637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:35:57.515348 637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:57.516573 637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:57.517866 637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:57.530714 643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:57.530678 645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:57.530840 642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:57.531563 637 server_base.cc:1034] running on GCE node
I20260618 22:35:57.532128 637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:57.532989 637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:57.534190 637 hybrid_clock.cc:648] HybridClock initialized: now 1781822157534140 us; error 41 us; skew 500 ppm
I20260618 22:35:57.534487 637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:57 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822157, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:57.537511 637 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:35:57.538923 637 webserver.cc:460] Webserver started at http://127.18.150.130:36495/ using document root <none> and password file <none>
I20260618 22:35:57.539623 637 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:57.539721 637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:57.539959 637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:35:57.542882 637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "18e7225647754aaa966d806975f09b11"
format_stamp: "Formatted at 2026-06-18 22:35:57 on dist-test-slave-h7jz"
server_key: "0c42ac44865880320234b4e0c97031e4"
server_key_iv: "c133ac88af74ba5d276c4b488cdff546"
server_key_version: "encryptionkey@0"
I20260618 22:35:57.543593 637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "18e7225647754aaa966d806975f09b11"
format_stamp: "Formatted at 2026-06-18 22:35:57 on dist-test-slave-h7jz"
server_key: "0c42ac44865880320234b4e0c97031e4"
server_key_iv: "c133ac88af74ba5d276c4b488cdff546"
server_key_version: "encryptionkey@0"
I20260618 22:35:57.548470 637 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:35:57.551697 652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:57.552932 637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:35:57.553071 637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "18e7225647754aaa966d806975f09b11"
format_stamp: "Formatted at 2026-06-18 22:35:57 on dist-test-slave-h7jz"
server_key: "0c42ac44865880320234b4e0c97031e4"
server_key_iv: "c133ac88af74ba5d276c4b488cdff546"
server_key_version: "encryptionkey@0"
I20260618 22:35:57.553241 637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:57.586038 637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:57.590148 637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:57.590390 637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:57.591288 637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:35:57.592550 637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:35:57.592618 637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:57.592733 637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:35:57.592778 637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:57.621316 637 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:46345
I20260618 22:35:57.621353 764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:46345 every 8 connection(s)
I20260618 22:35:57.622560 637 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:35:57.630854 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 637
I20260618 22:35:57.631031 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:35:57.631415 19034 external_mini_cluster.cc:1428] Setting key 2668866eac72aa18281e9ecae35a1bce
Jun 18 22:35:57 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822157, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:57.641278 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:57.624915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:58775 (local address 127.18.150.190:39461)
0618 22:35:57.625101 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:57.625105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:57.626253 (+ 1148us) server_negotiation.cc:407] Connection header received
0618 22:35:57.628025 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:57.628031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:57.628115 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:57.628201 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:57.629633 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:57.630419 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:57.631988 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:57.632492 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:57.632590 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:57.636137 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:57.636142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:57.636145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:57.636175 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:57.638591 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:57.639234 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:57.639240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:57.639243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:57.639367 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:57.639753 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:57.639758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:57.639761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:57.640101 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:57.640241 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:57.640873 (+ 632us) server_negotiation.cc:299] Negotiation successful
0618 22:35:57.641049 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:35:57.642241 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:57.625421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:58775)
0618 22:35:57.626108 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:57.626152 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:57.627752 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:57.628436 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:57.628444 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:57.628925 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:57.629457 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:57.629474 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:57.630564 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:57.630568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:57.631848 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:57.631860 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:57.632704 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:57.632711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:57.632971 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:57.633676 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:57.633702 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:57.635937 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:57.638719 (+ 2782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:57.638728 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:57.638745 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:57.639128 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:57.639463 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:57.639468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:57.639472 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:57.639655 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:57.640230 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:57.640236 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:57.640542 (+ 306us) client_negotiation.cc:769] Sending connection context
0618 22:35:57.640765 (+ 223us) client_negotiation.cc:240] Negotiation successful
0618 22:35:57.641142 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":261,"threads_started":1}
I20260618 22:35:57.643563 765 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39461
I20260618 22:35:57.643868 765 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:57.644681 765 heartbeater.cc:511] Master 127.18.150.190:39461 requested a full tablet report, sending...
I20260618 22:35:57.646165 442 ts_manager.cc:194] Registered new tserver with Master: 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345)
I20260618 22:35:57.646951 442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:58775
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:35:57.678490 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39461
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:35:57.992259 770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:57.992547 770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:57.992643 770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:57.999872 770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:57.999980 770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:58.000095 770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:35:58.009826 770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:58.011126 770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:58.012410 770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:58.025030 775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:58.025079 776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:58.025103 778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:58.025866 770 server_base.cc:1034] running on GCE node
I20260618 22:35:58.026501 770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:58.027423 770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:58.028644 770 hybrid_clock.cc:648] HybridClock initialized: now 1781822158028590 us; error 34 us; skew 500 ppm
I20260618 22:35:58.029016 770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:58 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822158, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:58.032106 770 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:35:58.033470 770 webserver.cc:460] Webserver started at http://127.18.150.131:34111/ using document root <none> and password file <none>
I20260618 22:35:58.034190 770 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:58.034297 770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:58.034575 770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:35:58.037433 770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "5d75adde76af449f953f0dfb3a9bcf5c"
format_stamp: "Formatted at 2026-06-18 22:35:58 on dist-test-slave-h7jz"
server_key: "a11ab6c802f4c1c8d3dc0c75b31b8b2f"
server_key_iv: "b73311f52a8d439800f3e0ec2b845849"
server_key_version: "encryptionkey@0"
I20260618 22:35:58.038250 770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "5d75adde76af449f953f0dfb3a9bcf5c"
format_stamp: "Formatted at 2026-06-18 22:35:58 on dist-test-slave-h7jz"
server_key: "a11ab6c802f4c1c8d3dc0c75b31b8b2f"
server_key_iv: "b73311f52a8d439800f3e0ec2b845849"
server_key_version: "encryptionkey@0"
I20260618 22:35:58.043423 770 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:35:58.046798 785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:58.048115 770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:35:58.048255 770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "5d75adde76af449f953f0dfb3a9bcf5c"
format_stamp: "Formatted at 2026-06-18 22:35:58 on dist-test-slave-h7jz"
server_key: "a11ab6c802f4c1c8d3dc0c75b31b8b2f"
server_key_iv: "b73311f52a8d439800f3e0ec2b845849"
server_key_version: "encryptionkey@0"
I20260618 22:35:58.048427 770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:58.071115 770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:58.075098 770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:58.075352 770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:58.076258 770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:35:58.077565 770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:35:58.077642 770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:58.077756 770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:35:58.077806 770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:58.107259 770 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:34617
I20260618 22:35:58.107295 897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:34617 every 8 connection(s)
I20260618 22:35:58.108623 770 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:35:58.110303 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 770
I20260618 22:35:58.110463 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:35:58.111044 19034 external_mini_cluster.cc:1428] Setting key 8b309ce228deebe2f9f6265f9931a105
Jun 18 22:35:58 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822158, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:35:58.127489 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.111132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:50413 (local address 127.18.150.190:39461)
0618 22:35:58.111292 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:58.111297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:58.112671 (+ 1374us) server_negotiation.cc:407] Connection header received
0618 22:35:58.114745 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:58.114750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:58.114834 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:58.114919 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:58.116344 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.117080 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.118605 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.119007 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.119113 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.122258 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:58.122262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:58.122266 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:58.122297 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:58.124617 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:58.125370 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:58.125375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:58.125377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:58.125496 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:58.125926 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:58.125931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:58.125934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:58.126327 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:58.126485 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:58.127051 (+ 566us) server_negotiation.cc:299] Negotiation successful
0618 22:35:58.127219 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:35:58.128388 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.111795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:50413)
0618 22:35:58.112534 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:58.112570 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:58.114504 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:58.115160 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:58.115168 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:58.115641 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:35:58.116193 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.116209 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.117216 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.117221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.118476 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.118487 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.119204 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.119210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.119434 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.120034 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:35:58.120056 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:35:58.122071 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:35:58.124736 (+ 2665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:58.124744 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:58.124760 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:58.125258 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:58.125597 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:35:58.125602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:35:58.125606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:35:58.125794 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:35:58.126469 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:35:58.126474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:35:58.126762 (+ 288us) client_negotiation.cc:769] Sending connection context
0618 22:35:58.126974 (+ 212us) client_negotiation.cc:240] Negotiation successful
0618 22:35:58.127348 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":277,"threads_started":1}
I20260618 22:35:58.129613 898 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39461
I20260618 22:35:58.129904 898 heartbeater.cc:464] Registering TS with master...
I20260618 22:35:58.130607 898 heartbeater.cc:511] Master 127.18.150.190:39461 requested a full tablet report, sending...
I20260618 22:35:58.131924 442 ts_manager.cc:194] Registered new tserver with Master: 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617)
I20260618 22:35:58.132687 442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:50413
I20260618 22:35:58.138458 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:35:58.154448 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.142796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33966 (local address 127.18.150.190:39461)
0618 22:35:58.142967 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:58.142973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:58.143398 (+ 425us) server_negotiation.cc:407] Connection header received
0618 22:35:58.143557 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:58.143562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:58.143637 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:58.143734 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:58.144964 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.145679 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.146958 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.147361 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.147460 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.149757 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:58.149762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:58.149765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:58.149794 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:58.152125 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:58.152767 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:58.152772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:58.152774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:58.152924 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:58.153306 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:58.153310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:58.153312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:58.153640 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:58.153820 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:58.154070 (+ 250us) server_negotiation.cc:299] Negotiation successful
0618 22:35:58.154227 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:35:58.158236 442 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:35:58.164757 442 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:35:58.170902 631 heartbeater.cc:503] Master 127.18.150.190:39461 was elected leader, sending a full tablet report...
I20260618 22:35:58.185712 911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.176623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39378 (local address 127.18.150.129:43581)
0618 22:35:58.177866 (+ 1243us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:58.177876 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:58.177904 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:35:58.178017 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:58.178024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:58.178282 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:58.178458 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:35:58.179347 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.180962 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.183450 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.184640 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.184837 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.185017 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:58.185244 (+ 227us) server_negotiation.cc:299] Negotiation successful
0618 22:35:58.185403 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1087,"thread_start_us":950,"threads_started":1}
I20260618 22:35:58.185748 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.176344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43581 (local address 127.0.0.1:39378)
0618 22:35:58.177185 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:58.177226 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:58.177381 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:58.178753 (+ 1372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:58.178760 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:58.178786 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:35:58.179138 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.179151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.181095 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.181101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.183310 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.183322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.184818 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.184824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.185014 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.185067 (+ 53us) client_negotiation.cc:769] Sending connection context
0618 22:35:58.185193 (+ 126us) client_negotiation.cc:240] Negotiation successful
0618 22:35:58.185372 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":1,"thread_start_us":429,"threads_started":1}
I20260618 22:35:58.186758 912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.177567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:46345 (local address 127.0.0.1:44566)
0618 22:35:58.178129 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:58.178145 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:58.178247 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:58.179768 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:58.179773 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:58.179797 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:35:58.180261 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.180273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.181857 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.181863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.184202 (+ 2339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.184216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.185935 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.185961 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.186133 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.186164 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:35:58.186229 (+ 65us) client_negotiation.cc:240] Negotiation successful
0618 22:35:58.186357 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":560,"thread_start_us":291,"threads_started":1}
I20260618 22:35:58.187022 914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.178212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:34617 (local address 127.0.0.1:54436)
0618 22:35:58.178648 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:58.178664 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:58.178754 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:58.180119 (+ 1365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:58.180124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:58.180150 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:35:58.180590 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.180602 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.182678 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.182683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.184590 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.184600 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.186302 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.186308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.186463 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.186497 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:35:58.186570 (+ 73us) client_negotiation.cc:240] Negotiation successful
0618 22:35:58.186692 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":816,"threads_started":1}
I20260618 22:35:58.187252 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.177891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44566 (local address 127.18.150.130:46345)
0618 22:35:58.179212 (+ 1321us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:58.179220 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:58.179243 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:35:58.179339 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:58.179345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:58.179565 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:58.180450 (+ 885us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:35:58.180656 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.181733 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.184365 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.185700 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.186598 (+ 898us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.186742 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:58.186812 (+ 70us) server_negotiation.cc:299] Negotiation successful
0618 22:35:58.186959 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1174,"thread_start_us":1014,"threads_started":1}
I20260618 22:35:58.187719 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.178788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54436 (local address 127.18.150.131:34617)
0618 22:35:58.179549 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:58.179557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:58.179587 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:35:58.179676 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:58.179683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:58.179899 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:58.180752 (+ 853us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:35:58.180974 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.182542 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.184723 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.186089 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.186303 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.186454 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:58.187264 (+ 810us) server_negotiation.cc:299] Negotiation successful
0618 22:35:58.187420 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":464,"threads_started":1}
I20260618 22:35:58.190114 566 tablet_service.cc:1467] Processing CreateTablet for tablet 6b9e1d3a8a66411bbb07ccb50ff50532 (DEFAULT_TABLE table=test-table [id=88f1daf27b6e4cc48521348bab355f0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:35:58.191027 700 tablet_service.cc:1467] Processing CreateTablet for tablet 6b9e1d3a8a66411bbb07ccb50ff50532 (DEFAULT_TABLE table=test-table [id=88f1daf27b6e4cc48521348bab355f0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:35:58.191603 566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b9e1d3a8a66411bbb07ccb50ff50532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:58.191282 833 tablet_service.cc:1467] Processing CreateTablet for tablet 6b9e1d3a8a66411bbb07ccb50ff50532 (DEFAULT_TABLE table=test-table [id=88f1daf27b6e4cc48521348bab355f0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:35:58.192076 700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b9e1d3a8a66411bbb07ccb50ff50532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:58.192569 833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b9e1d3a8a66411bbb07ccb50ff50532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:35:58.202705 916 tablet_bootstrap.cc:492] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: Bootstrap starting.
I20260618 22:35:58.205673 917 tablet_bootstrap.cc:492] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: Bootstrap starting.
I20260618 22:35:58.207090 916 tablet_bootstrap.cc:654] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: Neither blocks nor log segments found. Creating new log.
I20260618 22:35:58.207267 918 tablet_bootstrap.cc:492] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: Bootstrap starting.
I20260618 22:35:58.208405 916 log.cc:824] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:58.209072 917 tablet_bootstrap.cc:654] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: Neither blocks nor log segments found. Creating new log.
I20260618 22:35:58.210172 917 log.cc:824] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:58.210222 918 tablet_bootstrap.cc:654] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: Neither blocks nor log segments found. Creating new log.
I20260618 22:35:58.211372 916 tablet_bootstrap.cc:492] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: No bootstrap required, opened a new log
I20260618 22:35:58.211467 918 log.cc:824] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:58.211604 916 ts_tablet_manager.cc:1397] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260618 22:35:58.212968 917 tablet_bootstrap.cc:492] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: No bootstrap required, opened a new log
I20260618 22:35:58.213230 917 ts_tablet_manager.cc:1397] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260618 22:35:58.214336 918 tablet_bootstrap.cc:492] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: No bootstrap required, opened a new log
I20260618 22:35:58.214643 918 ts_tablet_manager.cc:1397] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260618 22:35:58.219216 916 raft_consensus.cc:348] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } }
I20260618 22:35:58.219838 916 raft_consensus.cc:374] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:58.219923 916 raft_consensus.cc:729] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e7225647754aaa966d806975f09b11, State: Initialized, Role: FOLLOWER
I20260618 22:35:58.220440 916 consensus_queue.cc:260] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } }
I20260618 22:35:58.221374 765 heartbeater.cc:503] Master 127.18.150.190:39461 was elected leader, sending a full tablet report...
I20260618 22:35:58.222020 916 ts_tablet_manager.cc:1428] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260618 22:35:58.223289 917 raft_consensus.cc:348] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } }
I20260618 22:35:58.223707 917 raft_consensus.cc:374] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:58.223804 917 raft_consensus.cc:729] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceeffefe2d6f4430971feaeef9474273, State: Initialized, Role: FOLLOWER
I20260618 22:35:58.224363 917 consensus_queue.cc:260] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } }
I20260618 22:35:58.224682 918 raft_consensus.cc:348] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } }
I20260618 22:35:58.225152 918 raft_consensus.cc:374] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:35:58.225281 918 raft_consensus.cc:729] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d75adde76af449f953f0dfb3a9bcf5c, State: Initialized, Role: FOLLOWER
I20260618 22:35:58.225878 918 consensus_queue.cc:260] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } }
I20260618 22:35:58.227119 917 ts_tablet_manager.cc:1428] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260618 22:35:58.228127 898 heartbeater.cc:503] Master 127.18.150.190:39461 was elected leader, sending a full tablet report...
I20260618 22:35:58.228643 918 ts_tablet_manager.cc:1428] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260618 22:35:58.360342 899 tablet.cc:2369] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:35:58.374377 766 tablet.cc:2369] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:35:58.393100 632 tablet.cc:2369] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:35:58.451210 923 raft_consensus.cc:482] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:35:58.451488 923 raft_consensus.cc:504] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } }
I20260618 22:35:58.453271 923 leader_election.cc:288] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345), 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617)
I20260618 22:35:58.459971 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.453627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:53081 (local address 127.18.150.130:46345)
0618 22:35:58.453801 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:58.453805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:58.454416 (+ 611us) server_negotiation.cc:407] Connection header received
0618 22:35:58.454650 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:58.454655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:58.454733 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:58.454839 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:35:58.455557 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.456302 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.458348 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.459132 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.459266 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.459342 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:58.459610 (+ 268us) server_negotiation.cc:299] Negotiation successful
0618 22:35:58.459733 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:35:58.460035 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.453573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:46345 (local address 127.18.150.129:53081)
0618 22:35:58.454314 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:58.454337 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:58.454491 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:58.455021 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:58.455026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:58.455072 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:35:58.455431 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.455443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.456423 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.456428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.458210 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.458223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.459317 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.459322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.459463 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.459486 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:35:58.459572 (+ 86us) client_negotiation.cc:240] Negotiation successful
0618 22:35:58.459696 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":398,"threads_started":1}
I20260618 22:35:58.460579 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.454686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:47367 (local address 127.18.150.131:34617)
0618 22:35:58.454932 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:58.454939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:58.455319 (+ 380us) server_negotiation.cc:407] Connection header received
0618 22:35:58.455475 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:58.455479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:58.455540 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:58.455635 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:35:58.456220 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.456971 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.458807 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.459592 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.459753 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.459845 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:58.460182 (+ 337us) server_negotiation.cc:299] Negotiation successful
0618 22:35:58.460323 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260618 22:35:58.461038 720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b9e1d3a8a66411bbb07ccb50ff50532" candidate_uuid: "ceeffefe2d6f4430971feaeef9474273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e7225647754aaa966d806975f09b11" is_pre_election: true
I20260618 22:35:58.461334 926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.454508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:34617 (local address 127.18.150.129:47367)
0618 22:35:58.455215 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0618 22:35:58.455235 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:35:58.455351 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:35:58.455669 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:35:58.455674 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:35:58.455702 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:35:58.456060 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.456070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.457115 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.457120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.458635 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:35:58.458649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.459826 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.459833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:35:58.459986 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.460010 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:35:58.460774 (+ 764us) client_negotiation.cc:240] Negotiation successful
0618 22:35:58.460914 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":1,"thread_start_us":357,"threads_started":1}
I20260618 22:35:58.461702 720 raft_consensus.cc:2454] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceeffefe2d6f4430971feaeef9474273 in term 0.
I20260618 22:35:58.462249 853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b9e1d3a8a66411bbb07ccb50ff50532" candidate_uuid: "ceeffefe2d6f4430971feaeef9474273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" is_pre_election: true
I20260618 22:35:58.462405 519 leader_election.cc:302] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e7225647754aaa966d806975f09b11, ceeffefe2d6f4430971feaeef9474273; no voters:
I20260618 22:35:58.462711 853 raft_consensus.cc:2454] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceeffefe2d6f4430971feaeef9474273 in term 0.
I20260618 22:35:58.462862 923 raft_consensus.cc:2781] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:35:58.463019 923 raft_consensus.cc:482] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:35:58.463141 923 raft_consensus.cc:3037] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:58.465442 923 raft_consensus.cc:504] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } }
I20260618 22:35:58.466120 923 leader_election.cc:288] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 1 election: Requested vote from peers 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345), 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617)
I20260618 22:35:58.466579 720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b9e1d3a8a66411bbb07ccb50ff50532" candidate_uuid: "ceeffefe2d6f4430971feaeef9474273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e7225647754aaa966d806975f09b11"
I20260618 22:35:58.466610 853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b9e1d3a8a66411bbb07ccb50ff50532" candidate_uuid: "ceeffefe2d6f4430971feaeef9474273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d75adde76af449f953f0dfb3a9bcf5c"
I20260618 22:35:58.466775 720 raft_consensus.cc:3037] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:58.466812 853 raft_consensus.cc:3037] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:35:58.468938 720 raft_consensus.cc:2454] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceeffefe2d6f4430971feaeef9474273 in term 1.
I20260618 22:35:58.469074 853 raft_consensus.cc:2454] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceeffefe2d6f4430971feaeef9474273 in term 1.
I20260618 22:35:58.469285 519 leader_election.cc:302] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e7225647754aaa966d806975f09b11, ceeffefe2d6f4430971feaeef9474273; no voters:
I20260618 22:35:58.469578 923 raft_consensus.cc:2781] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:35:58.470023 923 raft_consensus.cc:686] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 1 LEADER]: Becoming Leader. State: Replica: ceeffefe2d6f4430971feaeef9474273, State: Running, Role: LEADER
I20260618 22:35:58.470584 923 consensus_queue.cc:237] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } }
I20260618 22:35:58.474985 441 catalog_manager.cc:5495] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceeffefe2d6f4430971feaeef9474273 (127.18.150.129). New cstate: current_term: 1 leader_uuid: "ceeffefe2d6f4430971feaeef9474273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } health_report { overall_health: UNKNOWN } } }
I20260618 22:35:58.549669 911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.544717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39394 (local address 127.18.150.129:43581)
0618 22:35:58.544923 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:58.544928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:58.544946 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:35:58.545119 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:58.545123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:58.545212 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:58.545324 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:35:58.545615 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.546360 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.547779 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.548171 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.548253 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.548736 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:35:58.549230 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:35:58.549332 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:58.549369 (+ 37us) server_negotiation.cc:299] Negotiation successful
0618 22:35:58.549432 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:35:58.557461 720 raft_consensus.cc:1261] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11 [term 1 FOLLOWER]: Refusing update from remote peer ceeffefe2d6f4430971feaeef9474273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:35:58.557446 853 raft_consensus.cc:1261] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c [term 1 FOLLOWER]: Refusing update from remote peer ceeffefe2d6f4430971feaeef9474273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:35:58.558323 927 consensus_queue.cc:1035] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:35:58.558652 923 consensus_queue.cc:1035] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:35:58.570086 931 mvcc.cc:205] Tried to move back new op lower bound from 7298343561441890304 to 7298343561099169792. Current Snapshot: MvccSnapshot[applied={T|T < 7298343561441890304}]
I20260618 22:35:58.596347 442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:35:58.596649 442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:35:58.600276 442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Sending DeleteTablet for 3 replicas of tablet 6b9e1d3a8a66411bbb07ccb50ff50532
I20260618 22:35:58.601285 566 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b9e1d3a8a66411bbb07ccb50ff50532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:35:58 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:39378
I20260618 22:35:58.601483 833 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b9e1d3a8a66411bbb07ccb50ff50532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:35:58 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:54436
I20260618 22:35:58.601559 700 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b9e1d3a8a66411bbb07ccb50ff50532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:35:58 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:44566
I20260618 22:35:58.602210 940 tablet_replica.cc:330] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: stopping tablet replica
I20260618 22:35:58.602347 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 503
I20260618 22:35:58.602705 940 raft_consensus.cc:2229] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:35:58.603328 940 raft_consensus.cc:2258] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:35:58.609503 940 ts_tablet_manager.cc:1905] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:35:58.610978 942 tablet_replica.cc:330] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: stopping tablet replica
I20260618 22:35:58.611440 942 raft_consensus.cc:2229] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:35:58.611531 941 tablet_replica.cc:330] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: stopping tablet replica
I20260618 22:35:58.611959 942 raft_consensus.cc:2258] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:35:58.611990 941 raft_consensus.cc:2229] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:35:58.612578 941 raft_consensus.cc:2258] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:35:58.614770 941 ts_tablet_manager.cc:1905] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:35:58.616580 942 ts_tablet_manager.cc:1905] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:35:58.619388 940 ts_tablet_manager.cc:1918] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:35:58.619608 940 log.cc:1196] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/6b9e1d3a8a66411bbb07ccb50ff50532
I20260618 22:35:58.620215 940 ts_tablet_manager.cc:1939] T 6b9e1d3a8a66411bbb07ccb50ff50532 P ceeffefe2d6f4430971feaeef9474273: Deleting consensus metadata
I20260618 22:35:58.622212 426 catalog_manager.cc:4841] TS ceeffefe2d6f4430971feaeef9474273 (127.18.150.129:43581): tablet 6b9e1d3a8a66411bbb07ccb50ff50532 (table test-table [id=88f1daf27b6e4cc48521348bab355f0e]) successfully deleted
I20260618 22:35:58.624466 941 ts_tablet_manager.cc:1918] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:35:58.624651 941 log.cc:1196] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/6b9e1d3a8a66411bbb07ccb50ff50532
I20260618 22:35:58.625211 941 ts_tablet_manager.cc:1939] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 5d75adde76af449f953f0dfb3a9bcf5c: Deleting consensus metadata
I20260618 22:35:58.626936 426 catalog_manager.cc:4841] TS 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617): tablet 6b9e1d3a8a66411bbb07ccb50ff50532 (table test-table [id=88f1daf27b6e4cc48521348bab355f0e]) successfully deleted
I20260618 22:35:58.631191 942 ts_tablet_manager.cc:1918] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:35:58.631419 942 log.cc:1196] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/6b9e1d3a8a66411bbb07ccb50ff50532
I20260618 22:35:58.632033 942 ts_tablet_manager.cc:1939] T 6b9e1d3a8a66411bbb07ccb50ff50532 P 18e7225647754aaa966d806975f09b11: Deleting consensus metadata
Jun 18 22:35:58 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:35:58.633849 426 catalog_manager.cc:4841] TS 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345): tablet 6b9e1d3a8a66411bbb07ccb50ff50532 (table test-table [id=88f1daf27b6e4cc48521348bab355f0e]) successfully deleted
I20260618 22:35:58.641387 911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.603628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.18.150.129:43581)
0618 22:35:58.603812 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:58.603820 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:58.611818 (+ 7998us) server_negotiation.cc:407] Connection header received
0618 22:35:58.612127 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:58.612133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:58.612217 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:58.612294 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:58.614544 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.615759 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.617631 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.618362 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.618457 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.633656 (+ 15199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:58.633673 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:58.633690 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:58.633737 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:58.637626 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:58.638811 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:58.638820 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:58.638824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:58.639029 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:58.639470 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:58.639476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:58.639479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:58.639910 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:58.640239 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:58.640603 (+ 364us) server_negotiation.cc:299] Negotiation successful
0618 22:35:58.640834 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:35:58.642122 626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:58.740435 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 503
I20260618 22:35:58.755779 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 637
Jun 18 22:35:58 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:35:58.770335 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.756505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.18.150.130:46345)
0618 22:35:58.756679 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:58.756686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:58.756743 (+ 57us) server_negotiation.cc:407] Connection header received
0618 22:35:58.756950 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:58.756954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:58.757031 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:58.757112 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:58.758538 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.759226 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.760673 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.761064 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.761150 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.765199 (+ 4049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:58.765214 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:58.765229 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:58.765263 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:58.768057 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:58.768665 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:58.768672 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:58.768674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:58.768794 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:58.769073 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:58.769076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:58.769078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:58.769437 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:58.769650 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:58.769890 (+ 240us) server_negotiation.cc:299] Negotiation successful
0618 22:35:58.770082 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:35:58.770921 760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:58.857823 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 637
I20260618 22:35:58.870831 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 770
Jun 18 22:35:58 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:35:58.885399 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:35:58.871533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.18.150.131:34617)
0618 22:35:58.871719 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:35:58.871724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:35:58.871744 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:35:58.871907 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:35:58.871913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:35:58.871976 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:35:58.872074 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:35:58.873269 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.874006 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.875265 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:35:58.875793 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:35:58.875923 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:35:58.880027 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:35:58.880043 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:35:58.880060 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:35:58.880095 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:35:58.882708 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:58.883395 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:58.883403 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:58.883405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:58.883548 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:35:58.883910 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:35:58.883914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:35:58.883916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:35:58.884274 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:35:58.884561 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0618 22:35:58.884871 (+ 310us) server_negotiation.cc:299] Negotiation successful
0618 22:35:58.885093 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:35:58.886178 893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:58.972976 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 770
I20260618 22:35:58.985829 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 410
I20260618 22:35:58.986960 472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:35:59.097164 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 410
I20260618 22:35:59.114156 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39461
--webserver_interface=127.18.150.190
--webserver_port=42805
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:39461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:35:59.425779 948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:59.426110 948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:59.426245 948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:59.433449 948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:35:59.433578 948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:59.433652 948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:59.433733 948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:35:59.433794 948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:35:59.443475 948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:39461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=42805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:59.444739 948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:59.446130 948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:59.466630 954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:59.466584 955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:59.466542 957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:59.467484 948 server_base.cc:1034] running on GCE node
I20260618 22:35:59.468312 948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:59.469797 948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:59.470994 948 hybrid_clock.cc:648] HybridClock initialized: now 1781822159470933 us; error 52 us; skew 500 ppm
I20260618 22:35:59.471361 948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:59 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822159, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:59.474689 948 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:35:59.476241 948 webserver.cc:460] Webserver started at http://127.18.150.190:42805/ using document root <none> and password file <none>
I20260618 22:35:59.476963 948 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:59.477074 948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:59.482646 948 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:35:59.485353 964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:59.486721 948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:35:59.486887 948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "c7c524f35d3d4eca8221fd29b792a6b9"
format_stamp: "Formatted at 2026-06-18 22:35:56 on dist-test-slave-h7jz"
server_key: "b70daf49baa51442803614b38d2184cd"
server_key_iv: "60a16dd8d86cc54c00a320846d0f9cc1"
server_key_version: "encryptionkey@0"
I20260618 22:35:59.487963 948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:59.506803 948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:59.511018 948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:59.511267 948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:59.527668 948 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:39461
I20260618 22:35:59.527688 1015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:39461 every 8 connection(s)
I20260618 22:35:59.529121 948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:35:59.534065 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 948
I20260618 22:35:59.534794 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:43581
--local_ip_for_outbound_sockets=127.18.150.129
--tserver_master_addrs=127.18.150.190:39461
--webserver_port=43483
--webserver_interface=127.18.150.129
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:35:59.536123 1016 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:35:59.539669 1016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Bootstrap starting.
I20260618 22:35:59.567233 1016 log.cc:824] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:35:59.582715 1016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:35:59.583333 1016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Bootstrap complete.
I20260618 22:35:59.596848 1016 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:35:59.598250 1016 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7c524f35d3d4eca8221fd29b792a6b9, State: Initialized, Role: FOLLOWER
I20260618 22:35:59.599094 1016 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:35:59.599433 1016 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:35:59.599578 1016 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:35:59.599778 1016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:35:59.605647 1016 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:35:59.606317 1016 leader_election.cc:302] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7c524f35d3d4eca8221fd29b792a6b9; no voters:
I20260618 22:35:59.606947 1016 leader_election.cc:288] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:35:59.607118 1020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:35:59.608103 1020 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 2 LEADER]: Becoming Leader. State: Replica: c7c524f35d3d4eca8221fd29b792a6b9, State: Running, Role: LEADER
I20260618 22:35:59.608620 1020 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:35:59.609055 1016 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:35:59.610987 1021 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } } }
I20260618 22:35:59.610965 1022 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7c524f35d3d4eca8221fd29b792a6b9. Latest consensus state: current_term: 2 leader_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } } }
I20260618 22:35:59.611377 1022 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:59.611380 1021 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:35:59.612126 1026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:35:59.631043 1026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:35:59.632463 1026 catalog_manager.cc:1259] Loaded cluster ID: 6cda1eaba9f64238a81e7dec29335c02
I20260618 22:35:59.632618 1026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:35:59.635761 1026 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260618 22:35:59.652989 1026 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260618 22:35:59.661861 1026 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:35:59.663666 1026 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Loaded TSK: 0
I20260618 22:35:59.665393 1026 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:35:59.902037 1018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:35:59.902307 1018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:35:59.902426 1018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:35:59.909549 1018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:35:59.909653 1018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:35:59.909843 1018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:35:59.919803 1018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:43581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=43483
--tserver_master_addrs=127.18.150.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:35:59.921077 1018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:35:59.922413 1018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:35:59.935360 1043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:59.935752 1044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:35:59.936481 1046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:35:59.936858 1018 server_base.cc:1034] running on GCE node
I20260618 22:35:59.937441 1018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:35:59.938447 1018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:35:59.939662 1018 hybrid_clock.cc:648] HybridClock initialized: now 1781822159939607 us; error 47 us; skew 500 ppm
I20260618 22:35:59.939953 1018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:35:59 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822159, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:35:59.943198 1018 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:35:59.944612 1018 webserver.cc:460] Webserver started at http://127.18.150.129:43483/ using document root <none> and password file <none>
I20260618 22:35:59.945282 1018 fs_manager.cc:359] Metadata directory not provided
I20260618 22:35:59.945405 1018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:35:59.950779 1018 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:35:59.953436 1053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:59.954974 1018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:35:59.955122 1018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "ceeffefe2d6f4430971feaeef9474273"
format_stamp: "Formatted at 2026-06-18 22:35:57 on dist-test-slave-h7jz"
server_key: "8a745f92ca8f614560a142606547c688"
server_key_iv: "a11056c022482671461af1a918768374"
server_key_version: "encryptionkey@0"
I20260618 22:35:59.956086 1018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:35:59.970834 1018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:35:59.974834 1018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:35:59.975085 1018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:35:59.976092 1018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:35:59.977428 1018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:35:59.977530 1018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:35:59.977631 1018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:35:59.977699 1018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:00.009161 1018 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:43581
I20260618 22:36:00.009208 1165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:43581 every 8 connection(s)
I20260618 22:36:00.010695 1018 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:36:00.020146 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 1018
I20260618 22:36:00.020735 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:46345
--local_ip_for_outbound_sockets=127.18.150.130
--tserver_master_addrs=127.18.150.190:39461
--webserver_port=36495
--webserver_interface=127.18.150.130
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:36:00 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822159, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:36:00.032790 1169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.013652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:59003)
0618 22:36:00.014387 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:00.014425 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:00.016119 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:00.017492 (+ 1373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:00.017503 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:00.018141 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:36:00.018688 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.018706 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.020270 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.020276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.021537 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.021549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.022557 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.022565 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.022818 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.023425 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:36:00.023449 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:36:00.025509 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:36:00.028647 (+ 3138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:00.028656 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:00.028671 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:00.029084 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:00.029487 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:00.029492 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:00.029497 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:00.029676 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:00.030399 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:36:00.030407 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:36:00.030736 (+ 329us) client_negotiation.cc:769] Sending connection context
0618 22:36:00.030976 (+ 240us) client_negotiation.cc:240] Negotiation successful
0618 22:36:00.031513 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":299,"threads_started":1}
I20260618 22:36:00.032792 1168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.013439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:59003 (local address 127.18.150.190:39461)
0618 22:36:00.014079 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:00.014094 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:00.014560 (+ 466us) server_negotiation.cc:407] Connection header received
0618 22:36:00.016654 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:00.016668 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:00.016998 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:00.017326 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:00.018902 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.020112 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.021671 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.022344 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.022525 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.025747 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:00.025754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:00.025768 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:00.025807 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:00.028510 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:00.029207 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:00.029214 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:00.029219 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:00.029381 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:00.029777 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:00.029780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:00.029782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:00.030272 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:00.030460 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:00.031188 (+ 728us) server_negotiation.cc:299] Negotiation successful
0618 22:36:00.031565 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":217,"threads_started":1}
I20260618 22:36:00.036181 1166 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39461
I20260618 22:36:00.036518 1166 heartbeater.cc:464] Registering TS with master...
I20260618 22:36:00.037302 1166 heartbeater.cc:511] Master 127.18.150.190:39461 requested a full tablet report, sending...
I20260618 22:36:00.039446 981 ts_manager.cc:194] Registered new tserver with Master: ceeffefe2d6f4430971feaeef9474273 (127.18.150.129:43581)
I20260618 22:36:00.041025 981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:59003
W20260618 22:36:00.328263 1170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:00.328548 1170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:00.328675 1170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:00.335857 1170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:00.335978 1170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:00.336128 1170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:36:00.348788 1170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:46345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=36495
--tserver_master_addrs=127.18.150.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:00.350178 1170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:00.351418 1170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:00.363418 1176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:00.363461 1178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:00.363484 1175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:00.364322 1170 server_base.cc:1034] running on GCE node
I20260618 22:36:00.364821 1170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:00.365643 1170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:00.366842 1170 hybrid_clock.cc:648] HybridClock initialized: now 1781822160366795 us; error 32 us; skew 500 ppm
I20260618 22:36:00.367153 1170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:00 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822160, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:00.370044 1170 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:36:00.371376 1170 webserver.cc:460] Webserver started at http://127.18.150.130:36495/ using document root <none> and password file <none>
I20260618 22:36:00.371987 1170 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:00.372107 1170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:00.377133 1170 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:36:00.379726 1185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:00.381042 1170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:36:00.381188 1170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "18e7225647754aaa966d806975f09b11"
format_stamp: "Formatted at 2026-06-18 22:35:57 on dist-test-slave-h7jz"
server_key: "0c42ac44865880320234b4e0c97031e4"
server_key_iv: "c133ac88af74ba5d276c4b488cdff546"
server_key_version: "encryptionkey@0"
I20260618 22:36:00.382157 1170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:00.413321 1170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:00.417589 1170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:00.417846 1170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:00.418862 1170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:36:00.420208 1170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:36:00.420321 1170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:00.420418 1170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:36:00.420475 1170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:00.450727 1170 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:46345
I20260618 22:36:00.450799 1297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:46345 every 8 connection(s)
I20260618 22:36:00.452157 1170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:36:00.461272 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 1170
I20260618 22:36:00.461848 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:34617
--local_ip_for_outbound_sockets=127.18.150.131
--tserver_master_addrs=127.18.150.190:39461
--webserver_port=34111
--webserver_interface=127.18.150.131
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:36:00 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822160, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:36:00.472028 1168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.454755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:42447 (local address 127.18.150.190:39461)
0618 22:36:00.454914 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:00.454919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:00.456181 (+ 1262us) server_negotiation.cc:407] Connection header received
0618 22:36:00.457990 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:00.457995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:00.458082 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:00.458168 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:00.459710 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.460403 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.462080 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.462705 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.462833 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.466071 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:00.466076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:00.466080 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:00.466118 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:00.468854 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:00.469778 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:00.469782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:00.469784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:00.469970 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:00.470398 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:00.470401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:00.470403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:00.470675 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:00.470777 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:00.471594 (+ 817us) server_negotiation.cc:299] Negotiation successful
0618 22:36:00.471774 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:36:00.473392 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.455310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:42447)
0618 22:36:00.455994 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:00.456057 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:00.457686 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:00.458412 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:00.458420 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:00.458913 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:36:00.459545 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.459565 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.460561 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.460567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.461874 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.461887 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.462907 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.462912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.463129 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.463754 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:36:00.463776 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:36:00.465824 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:36:00.469029 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:00.469039 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:00.469057 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:00.469655 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:00.470072 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:00.470078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:00.470081 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:00.470310 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:00.470790 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:36:00.470798 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:36:00.471213 (+ 415us) client_negotiation.cc:769] Sending connection context
0618 22:36:00.471514 (+ 301us) client_negotiation.cc:240] Negotiation successful
0618 22:36:00.472057 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":264,"threads_started":1}
I20260618 22:36:00.474843 1298 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39461
I20260618 22:36:00.475140 1298 heartbeater.cc:464] Registering TS with master...
I20260618 22:36:00.475871 1298 heartbeater.cc:511] Master 127.18.150.190:39461 requested a full tablet report, sending...
I20260618 22:36:00.477269 981 ts_manager.cc:194] Registered new tserver with Master: 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345)
I20260618 22:36:00.478137 981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:42447
W20260618 22:36:00.775923 1301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:00.776172 1301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:00.776299 1301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:00.783210 1301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:00.783309 1301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:00.783456 1301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:36:00.792742 1301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:34617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=34111
--tserver_master_addrs=127.18.150.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:00.793895 1301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:00.795151 1301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:00.807335 1306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:00.807426 1309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:00.807452 1307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:00.808166 1301 server_base.cc:1034] running on GCE node
I20260618 22:36:00.808704 1301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:00.809585 1301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:00.810796 1301 hybrid_clock.cc:648] HybridClock initialized: now 1781822160810726 us; error 60 us; skew 500 ppm
I20260618 22:36:00.811108 1301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:00 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822160, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:00.814077 1301 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:36:00.815407 1301 webserver.cc:460] Webserver started at http://127.18.150.131:34111/ using document root <none> and password file <none>
I20260618 22:36:00.816041 1301 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:00.816165 1301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:00.821096 1301 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:36:00.823616 1316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:00.824815 1301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:36:00.824956 1301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "5d75adde76af449f953f0dfb3a9bcf5c"
format_stamp: "Formatted at 2026-06-18 22:35:58 on dist-test-slave-h7jz"
server_key: "a11ab6c802f4c1c8d3dc0c75b31b8b2f"
server_key_iv: "b73311f52a8d439800f3e0ec2b845849"
server_key_version: "encryptionkey@0"
I20260618 22:36:00.825839 1301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:00.837004 1301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:00.840845 1301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:00.841078 1301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:00.841933 1301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:36:00.843137 1301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:36:00.843230 1301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:00.843325 1301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:36:00.843389 1301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:00.872627 1301 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:34617
I20260618 22:36:00.872700 1428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:34617 every 8 connection(s)
I20260618 22:36:00.874166 1301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:36:00.874557 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 1301
Jun 18 22:36:00 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822160, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:36:00.890833 1168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.875526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52130 (local address 127.18.150.190:39461)
0618 22:36:00.875698 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:00.875703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:00.877365 (+ 1662us) server_negotiation.cc:407] Connection header received
0618 22:36:00.877513 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:00.877518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:00.877597 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:00.877705 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:00.879505 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.880216 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.882054 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.882461 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.882571 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.885581 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:00.885586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:00.885590 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:00.885621 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:00.888268 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:00.888979 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:00.888986 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:00.888989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:00.889142 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:00.889496 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:00.889502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:00.889505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:00.889828 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:00.890101 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:00.890363 (+ 262us) server_negotiation.cc:299] Negotiation successful
0618 22:36:00.890527 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":53}
I20260618 22:36:00.895856 1432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.876997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:60825 (local address 127.18.150.190:39461)
0618 22:36:00.877491 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:00.877497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:00.878725 (+ 1228us) server_negotiation.cc:407] Connection header received
0618 22:36:00.880720 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:00.880723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:00.880779 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:00.880855 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:00.882590 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.883559 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.885737 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.886554 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.886683 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.890232 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:00.890238 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:00.890243 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:00.890277 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:00.893329 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:00.894021 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:00.894025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:00.894029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:00.894132 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:00.894484 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:00.894487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:00.894489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:00.894739 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:00.894838 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:00.895443 (+ 605us) server_negotiation.cc:299] Negotiation successful
0618 22:36:00.895597 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":382,"thread_start_us":262,"threads_started":1}
I20260618 22:36:00.896766 1433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.877714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:60825)
0618 22:36:00.878564 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:00.878614 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:00.880470 (+ 1856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:00.881100 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:00.881111 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:00.881771 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:36:00.882433 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.882450 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.883711 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.883717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.885624 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.885635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.886740 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.886744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.886940 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.887598 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:36:00.887619 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:36:00.890015 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:36:00.893468 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:00.893478 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:00.893492 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:00.893889 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:00.894211 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:00.894214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:00.894216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:00.894405 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:00.894839 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:36:00.894844 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:36:00.895163 (+ 319us) client_negotiation.cc:769] Sending connection context
0618 22:36:00.895381 (+ 218us) client_negotiation.cc:240] Negotiation successful
0618 22:36:00.895767 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":328,"threads_started":1}
I20260618 22:36:00.897991 1429 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39461
I20260618 22:36:00.898263 1429 heartbeater.cc:464] Registering TS with master...
I20260618 22:36:00.898926 1429 heartbeater.cc:511] Master 127.18.150.190:39461 requested a full tablet report, sending...
I20260618 22:36:00.900375 981 ts_manager.cc:194] Registered new tserver with Master: 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617)
I20260618 22:36:00.901154 981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:60825
I20260618 22:36:00.905637 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:36:00.912014 1432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.906495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52142 (local address 127.18.150.190:39461)
0618 22:36:00.906658 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:00.906664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:00.907096 (+ 432us) server_negotiation.cc:407] Connection header received
0618 22:36:00.907282 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:00.907285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:00.907403 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:00.907537 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:36:00.907929 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.908599 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.909999 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.910415 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.910533 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.911003 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:36:00.911463 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:36:00.911608 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:00.911693 (+ 85us) server_negotiation.cc:299] Negotiation successful
0618 22:36:00.911773 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:36:00.912833 981 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:36:00.919708 981 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:36:00.942176 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.931024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43581 (local address 127.0.0.1:40686)
0618 22:36:00.932453 (+ 1429us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:00.932494 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:00.932658 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:00.933585 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:00.933593 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:00.933626 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:00.934042 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.934055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.936081 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.936087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.938812 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.938829 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.941114 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.941122 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.941346 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.941402 (+ 56us) client_negotiation.cc:769] Sending connection context
0618 22:36:00.941534 (+ 132us) client_negotiation.cc:240] Negotiation successful
0618 22:36:00.941743 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1233,"thread_start_us":1092,"threads_started":1}
I20260618 22:36:00.943197 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.931216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.18.150.129:43581)
0618 22:36:00.931784 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:00.931793 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:00.932597 (+ 804us) server_negotiation.cc:407] Connection header received
0618 22:36:00.932849 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:00.932858 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:00.933110 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:00.933299 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:00.934266 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.935905 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.939022 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.940590 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.940761 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.940950 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:00.942690 (+ 1740us) server_negotiation.cc:299] Negotiation successful
0618 22:36:00.942890 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":337,"threads_started":1}
I20260618 22:36:00.948063 1101 tablet_service.cc:1467] Processing CreateTablet for tablet 4f76619e34094466a28e7b77d79aadff (DEFAULT_TABLE table=test-table [id=2e9c1974a79f4154a153d89b8005ce39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:36:00.949527 1101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f76619e34094466a28e7b77d79aadff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:36:00.951140 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.933345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:46345 (local address 127.0.0.1:47506)
0618 22:36:00.942414 (+ 9069us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:00.942430 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:00.942525 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:00.944756 (+ 2231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:00.944761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:00.944785 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:00.944902 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.944909 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.946689 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.946694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.948557 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.948572 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.950335 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.950341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.950525 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.950560 (+ 35us) client_negotiation.cc:769] Sending connection context
0618 22:36:00.950664 (+ 104us) client_negotiation.cc:240] Negotiation successful
0618 22:36:00.950801 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8986,"mutex_wait_us":1,"thread_start_us":10556,"threads_started":1}
I20260618 22:36:00.951207 1444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.934028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47506 (local address 127.18.150.130:46345)
0618 22:36:00.943855 (+ 9827us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:00.943864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:00.943900 (+ 36us) server_negotiation.cc:407] Connection header received
0618 22:36:00.944005 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:00.944014 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:00.944274 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:00.944474 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:00.945050 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.946550 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.948715 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.950171 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.950367 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.950536 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:00.950703 (+ 167us) server_negotiation.cc:299] Negotiation successful
0618 22:36:00.950864 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9643,"thread_start_us":9502,"threads_started":1}
I20260618 22:36:00.951380 1441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.932777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:34617 (local address 127.0.0.1:47158)
0618 22:36:00.933274 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:00.933293 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:00.933388 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:00.945066 (+ 11678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:00.945069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:00.945092 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:00.945435 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.945447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.947136 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.947141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.948802 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:00.948824 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.950649 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.950655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:00.950845 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.950877 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:36:00.950941 (+ 64us) client_negotiation.cc:240] Negotiation successful
0618 22:36:00.951072 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":268,"threads_started":1}
I20260618 22:36:00.952219 1442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:00.933081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.18.150.131:34617)
0618 22:36:00.944471 (+ 11390us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:00.944479 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:00.944521 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:36:00.944627 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:00.944634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:00.944868 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:00.945080 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:00.945634 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.947008 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.949062 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:00.950416 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:00.951548 (+ 1132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:00.951704 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:00.951772 (+ 68us) server_negotiation.cc:299] Negotiation successful
0618 22:36:00.951932 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11228,"thread_start_us":11087,"threads_started":1}
I20260618 22:36:00.955329 1233 tablet_service.cc:1467] Processing CreateTablet for tablet 4f76619e34094466a28e7b77d79aadff (DEFAULT_TABLE table=test-table [id=2e9c1974a79f4154a153d89b8005ce39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:36:00.955319 1364 tablet_service.cc:1467] Processing CreateTablet for tablet 4f76619e34094466a28e7b77d79aadff (DEFAULT_TABLE table=test-table [id=2e9c1974a79f4154a153d89b8005ce39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:36:00.956734 1364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f76619e34094466a28e7b77d79aadff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:36:00.956734 1233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f76619e34094466a28e7b77d79aadff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:36:00.963228 1445 tablet_bootstrap.cc:492] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Bootstrap starting.
I20260618 22:36:00.966248 1445 tablet_bootstrap.cc:654] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Neither blocks nor log segments found. Creating new log.
I20260618 22:36:00.967284 1445 log.cc:824] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:00.970790 1445 tablet_bootstrap.cc:492] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: No bootstrap required, opened a new log
I20260618 22:36:00.971036 1445 ts_tablet_manager.cc:1397] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260618 22:36:00.971841 1448 tablet_bootstrap.cc:492] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c: Bootstrap starting.
I20260618 22:36:00.972584 1447 tablet_bootstrap.cc:492] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11: Bootstrap starting.
I20260618 22:36:00.975935 1447 tablet_bootstrap.cc:654] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11: Neither blocks nor log segments found. Creating new log.
I20260618 22:36:00.976830 1448 tablet_bootstrap.cc:654] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c: Neither blocks nor log segments found. Creating new log.
I20260618 22:36:00.977130 1447 log.cc:824] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:00.977834 1448 log.cc:824] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:00.979876 1447 tablet_bootstrap.cc:492] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11: No bootstrap required, opened a new log
I20260618 22:36:00.980135 1448 tablet_bootstrap.cc:492] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c: No bootstrap required, opened a new log
I20260618 22:36:00.980139 1447 ts_tablet_manager.cc:1397] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260618 22:36:00.980348 1448 ts_tablet_manager.cc:1397] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260618 22:36:00.980504 1445 raft_consensus.cc:348] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:00.980957 1445 raft_consensus.cc:374] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:36:00.981055 1445 raft_consensus.cc:729] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceeffefe2d6f4430971feaeef9474273, State: Initialized, Role: FOLLOWER
I20260618 22:36:00.981652 1445 consensus_queue.cc:260] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:00.982733 1166 heartbeater.cc:503] Master 127.18.150.190:39461 was elected leader, sending a full tablet report...
I20260618 22:36:00.983340 1445 ts_tablet_manager.cc:1428] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260618 22:36:00.988296 1447 raft_consensus.cc:348] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:00.988633 1447 raft_consensus.cc:374] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:36:00.988729 1447 raft_consensus.cc:729] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e7225647754aaa966d806975f09b11, State: Initialized, Role: FOLLOWER
I20260618 22:36:00.989207 1447 consensus_queue.cc:260] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:00.990163 1298 heartbeater.cc:503] Master 127.18.150.190:39461 was elected leader, sending a full tablet report...
I20260618 22:36:00.990301 1448 raft_consensus.cc:348] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:00.990576 1447 ts_tablet_manager.cc:1428] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11: Time spent starting tablet: real 0.010s user 0.012s sys 0.000s
I20260618 22:36:00.990676 1448 raft_consensus.cc:374] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:36:00.990758 1448 raft_consensus.cc:729] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d75adde76af449f953f0dfb3a9bcf5c, State: Initialized, Role: FOLLOWER
I20260618 22:36:00.991303 1448 consensus_queue.cc:260] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:00.992359 1429 heartbeater.cc:503] Master 127.18.150.190:39461 was elected leader, sending a full tablet report...
I20260618 22:36:00.992828 1448 ts_tablet_manager.cc:1428] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
W20260618 22:36:01.012627 1167 tablet.cc:2369] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:36:01.055160 1451 raft_consensus.cc:482] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:36:01.055466 1451 raft_consensus.cc:504] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:01.057035 1451 leader_election.cc:288] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617), 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345)
I20260618 22:36:01.061961 1453 raft_consensus.cc:482] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:36:01.062253 1453 raft_consensus.cc:504] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:01.064879 1442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.057481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:39435 (local address 127.18.150.131:34617)
0618 22:36:01.057699 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:01.057706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:01.058190 (+ 484us) server_negotiation.cc:407] Connection header received
0618 22:36:01.058457 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:01.058463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:01.058542 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:01.058599 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:01.059290 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.060201 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.062268 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.063561 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.063695 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.063767 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:01.064497 (+ 730us) server_negotiation.cc:299] Negotiation successful
0618 22:36:01.064634 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:36:01.064960 1454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.057358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:34617 (local address 127.18.150.129:39435)
0618 22:36:01.058095 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:01.058112 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:01.058266 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:01.058762 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:01.058767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:01.058795 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:01.059154 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:01.059166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.060337 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.060342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:01.062023 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:01.062035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.063919 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.063926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:01.064366 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.064396 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:36:01.064469 (+ 73us) client_negotiation.cc:240] Negotiation successful
0618 22:36:01.064601 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":223,"thread_start_us":303,"threads_started":1}
I20260618 22:36:01.065577 1444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.058303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:41801 (local address 127.18.150.130:46345)
0618 22:36:01.058518 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:01.058524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:01.058797 (+ 273us) server_negotiation.cc:407] Connection header received
0618 22:36:01.058943 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:01.058948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:01.059020 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:01.059108 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:01.059750 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.060541 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.062382 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.063549 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.063695 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.063767 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:01.065227 (+ 1460us) server_negotiation.cc:299] Negotiation successful
0618 22:36:01.065358 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:36:01.065861 1384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f76619e34094466a28e7b77d79aadff" candidate_uuid: "ceeffefe2d6f4430971feaeef9474273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" is_pre_election: true
I20260618 22:36:01.066305 1455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.058144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:46345 (local address 127.18.150.129:41801)
0618 22:36:01.058705 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:01.058718 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:01.058818 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:01.059148 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:01.059151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:01.059168 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:01.059608 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:01.059621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.060650 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.060655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:01.062177 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:01.062191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.063954 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.063959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:01.064155 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.064196 (+ 41us) client_negotiation.cc:769] Sending connection context
0618 22:36:01.065803 (+ 1607us) client_negotiation.cc:240] Negotiation successful
0618 22:36:01.065957 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":1,"thread_start_us":274,"threads_started":1}
I20260618 22:36:01.066359 1384 raft_consensus.cc:2454] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceeffefe2d6f4430971feaeef9474273 in term 0.
I20260618 22:36:01.067061 1054 leader_election.cc:302] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d75adde76af449f953f0dfb3a9bcf5c, ceeffefe2d6f4430971feaeef9474273; no voters:
I20260618 22:36:01.067476 1451 raft_consensus.cc:2781] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:36:01.067615 1451 raft_consensus.cc:482] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:36:01.067711 1451 raft_consensus.cc:3037] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:36:01.070108 1453 leader_election.cc:288] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceeffefe2d6f4430971feaeef9474273 (127.18.150.129:43581), 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345)
I20260618 22:36:01.071022 1451 raft_consensus.cc:504] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:01.072189 1384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f76619e34094466a28e7b77d79aadff" candidate_uuid: "ceeffefe2d6f4430971feaeef9474273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d75adde76af449f953f0dfb3a9bcf5c"
I20260618 22:36:01.072389 1384 raft_consensus.cc:3037] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:36:01.073833 1433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.064048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43581 (local address 127.18.150.131:33313)
0618 22:36:01.064203 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:01.064221 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:01.064315 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:01.065641 (+ 1326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:01.065648 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:01.065685 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:01.065918 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:01.065933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.068281 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.068288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:01.070491 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:01.070511 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.072838 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.072846 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:01.073140 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.073184 (+ 44us) client_negotiation.cc:769] Sending connection context
0618 22:36:01.073290 (+ 106us) client_negotiation.cc:240] Negotiation successful
0618 22:36:01.073464 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260618 22:36:01.074602 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.064207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:33313 (local address 127.18.150.129:43581)
0618 22:36:01.064355 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:01.064360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:01.064376 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:36:01.064423 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:01.064427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:01.064486 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:01.064589 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:01.066763 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.067762 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.070787 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.072293 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.072473 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.072561 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:01.074140 (+ 1579us) server_negotiation.cc:299] Negotiation successful
0618 22:36:01.074292 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:36:01.075949 1384 raft_consensus.cc:2454] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceeffefe2d6f4430971feaeef9474273 in term 1.
I20260618 22:36:01.076392 1054 leader_election.cc:302] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d75adde76af449f953f0dfb3a9bcf5c, ceeffefe2d6f4430971feaeef9474273; no voters:
I20260618 22:36:01.077266 1458 raft_consensus.cc:2781] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:36:01.077549 1451 leader_election.cc:288] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 1 election: Requested vote from peers 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617), 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345)
I20260618 22:36:01.077700 1121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f76619e34094466a28e7b77d79aadff" candidate_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceeffefe2d6f4430971feaeef9474273" is_pre_election: true
I20260618 22:36:01.079079 1252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f76619e34094466a28e7b77d79aadff" candidate_uuid: "ceeffefe2d6f4430971feaeef9474273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e7225647754aaa966d806975f09b11"
I20260618 22:36:01.079026 1253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f76619e34094466a28e7b77d79aadff" candidate_uuid: "ceeffefe2d6f4430971feaeef9474273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e7225647754aaa966d806975f09b11" is_pre_election: true
I20260618 22:36:01.079579 1253 raft_consensus.cc:2454] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceeffefe2d6f4430971feaeef9474273 in term 0.
I20260618 22:36:01.080199 1458 raft_consensus.cc:686] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 LEADER]: Becoming Leader. State: Replica: ceeffefe2d6f4430971feaeef9474273, State: Running, Role: LEADER
I20260618 22:36:01.080861 1458 consensus_queue.cc:237] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:01.083434 1444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.064856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:39155 (local address 127.18.150.130:46345)
0618 22:36:01.065763 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:01.065769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:01.075449 (+ 9680us) server_negotiation.cc:407] Connection header received
0618 22:36:01.075540 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:01.075547 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:01.075634 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:01.075724 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:01.076513 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.079464 (+ 2951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.081599 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.082434 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.082557 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.082605 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:01.083039 (+ 434us) server_negotiation.cc:299] Negotiation successful
0618 22:36:01.083175 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":13257,"threads_started":1}
I20260618 22:36:01.083492 1433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.064596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:46345 (local address 127.18.150.131:39155)
0618 22:36:01.075152 (+ 10556us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:01.075174 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:01.075320 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:01.075986 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:01.075992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:01.076023 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:01.076318 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:01.076334 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.079677 (+ 3343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.079683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:01.081457 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:01.081472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.082614 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.082621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:01.082823 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.082861 (+ 38us) client_negotiation.cc:769] Sending connection context
0618 22:36:01.082973 (+ 112us) client_negotiation.cc:240] Negotiation successful
0618 22:36:01.083122 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10442,"thread_start_us":12012,"threads_started":1}
I20260618 22:36:01.085011 1253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f76619e34094466a28e7b77d79aadff" candidate_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e7225647754aaa966d806975f09b11" is_pre_election: true
I20260618 22:36:01.085273 1253 raft_consensus.cc:2454] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d75adde76af449f953f0dfb3a9bcf5c in term 0.
I20260618 22:36:01.085794 1317 leader_election.cc:302] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18e7225647754aaa966d806975f09b11, 5d75adde76af449f953f0dfb3a9bcf5c; no voters: ceeffefe2d6f4430971feaeef9474273
I20260618 22:36:01.086169 1453 raft_consensus.cc:2741] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260618 22:36:01.087170 981 catalog_manager.cc:5495] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceeffefe2d6f4430971feaeef9474273 (127.18.150.129). New cstate: current_term: 1 leader_uuid: "ceeffefe2d6f4430971feaeef9474273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } health_report { overall_health: UNKNOWN } } }
I20260618 22:36:01.108304 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.103389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.18.150.129:43581)
0618 22:36:01.103566 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:01.103571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:01.103643 (+ 72us) server_negotiation.cc:407] Connection header received
0618 22:36:01.103808 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:01.103812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:01.103893 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:01.104003 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:36:01.104320 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.105045 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.106395 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.106781 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.106871 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.107316 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:36:01.107814 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:36:01.107916 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:01.107954 (+ 38us) server_negotiation.cc:299] Negotiation successful
0618 22:36:01.108016 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:36:01.115875 1253 raft_consensus.cc:3037] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:36:01.116091 1384 raft_consensus.cc:1261] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 1 FOLLOWER]: Refusing update from remote peer ceeffefe2d6f4430971feaeef9474273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:36:01.116894 1459 consensus_queue.cc:1035] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:36:01.118579 1253 raft_consensus.cc:1261] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11 [term 1 FOLLOWER]: Refusing update from remote peer ceeffefe2d6f4430971feaeef9474273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:36:01.119280 1459 consensus_queue.cc:1035] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:36:01.128700 1462 mvcc.cc:205] Tried to move back new op lower bound from 7298343571922214912 to 7298343571792146432. Current Snapshot: MvccSnapshot[applied={T|T < 7298343571922214912}]
I20260618 22:36:01.134258 1464 mvcc.cc:205] Tried to move back new op lower bound from 7298343571922214912 to 7298343571792146432. Current Snapshot: MvccSnapshot[applied={T|T < 7298343571922214912}]
I20260618 22:36:01.144915 980 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:36:01.145165 980 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:36:01.148437 980 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Sending DeleteTablet for 3 replicas of tablet 4f76619e34094466a28e7b77d79aadff
I20260618 22:36:01.149644 1364 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f76619e34094466a28e7b77d79aadff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:36:01 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:47158
I20260618 22:36:01.149799 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 1018
I20260618 22:36:01.149745 1233 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f76619e34094466a28e7b77d79aadff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:36:01 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:47506
I20260618 22:36:01.149744 1101 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f76619e34094466a28e7b77d79aadff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:36:01 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:40686
I20260618 22:36:01.164964 1471 tablet_replica.cc:330] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c: stopping tablet replica
I20260618 22:36:01.165450 1471 raft_consensus.cc:2229] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:36:01.166000 1471 raft_consensus.cc:2258] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:36:01.167693 1473 tablet_replica.cc:330] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11: stopping tablet replica
I20260618 22:36:01.167733 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.150622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40718 (local address 127.18.150.129:43581)
0618 22:36:01.150814 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:01.150821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:01.150937 (+ 116us) server_negotiation.cc:407] Connection header received
0618 22:36:01.151119 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:01.151125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:01.151210 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:01.151306 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:01.153146 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.154367 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.156340 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.157049 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.157171 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.160691 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:01.160705 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:01.160725 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:01.160766 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:01.164756 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:01.165605 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:01.165614 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:01.165618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:01.165800 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:01.166215 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:01.166219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:01.166222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:01.166675 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:01.166975 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:01.167189 (+ 214us) server_negotiation.cc:299] Negotiation successful
0618 22:36:01.167414 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:36:01.168102 1473 raft_consensus.cc:2229] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:36:01.168308 1471 ts_tablet_manager.cc:1905] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:36:01.168445 1161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:01.168599 1473 raft_consensus.cc:2258] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:36:01.171006 1473 ts_tablet_manager.cc:1905] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:36:01.174111 1471 ts_tablet_manager.cc:1918] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:36:01.174257 1471 log.cc:1196] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/4f76619e34094466a28e7b77d79aadff
I20260618 22:36:01.174726 1471 ts_tablet_manager.cc:1939] T 4f76619e34094466a28e7b77d79aadff P 5d75adde76af449f953f0dfb3a9bcf5c: Deleting consensus metadata
I20260618 22:36:01.176208 965 catalog_manager.cc:4841] TS 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617): tablet 4f76619e34094466a28e7b77d79aadff (table test-table [id=2e9c1974a79f4154a153d89b8005ce39]) successfully deleted
I20260618 22:36:01.177495 1473 ts_tablet_manager.cc:1918] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:36:01.177685 1473 log.cc:1196] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/4f76619e34094466a28e7b77d79aadff
I20260618 22:36:01.178210 1473 ts_tablet_manager.cc:1939] T 4f76619e34094466a28e7b77d79aadff P 18e7225647754aaa966d806975f09b11: Deleting consensus metadata
I20260618 22:36:01.179906 965 catalog_manager.cc:4841] TS 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345): tablet 4f76619e34094466a28e7b77d79aadff (table test-table [id=2e9c1974a79f4154a153d89b8005ce39]) successfully deleted
I20260618 22:36:01.266170 1472 tablet_replica.cc:330] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: stopping tablet replica
I20260618 22:36:01.266397 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 1018
W20260618 22:36:01.281426 965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.150.129:43581 (error 108)
I20260618 22:36:01.281502 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 1170
I20260618 22:36:01.283301 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.282824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43581 (local address 127.0.0.1:40734)
0618 22:36:01.283008 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:01.283119 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260618 22:36:01.283543 965 catalog_manager.cc:4584] TS ceeffefe2d6f4430971feaeef9474273 (127.18.150.129:43581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f76619e34094466a28e7b77d79aadff: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43581: connect: Connection refused (error 111)
I20260618 22:36:01.298236 1444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.282187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47520 (local address 127.18.150.130:46345)
0618 22:36:01.282361 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:01.282365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:01.282506 (+ 141us) server_negotiation.cc:407] Connection header received
0618 22:36:01.282685 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:01.282690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:01.282785 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:01.282896 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:01.284303 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.285018 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.286319 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.286777 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.286870 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.293081 (+ 6211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:01.293095 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:01.293110 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:01.293148 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:01.295745 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:01.296408 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:01.296415 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:01.296417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:01.296536 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:01.296890 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:01.296892 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:01.296894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:01.297230 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:01.297449 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:01.297730 (+ 281us) server_negotiation.cc:299] Negotiation successful
0618 22:36:01.297896 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:36:01.298827 1293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:01.337244 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.336694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43581 (local address 127.0.0.1:40740)
0618 22:36:01.336931 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:01.337070 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260618 22:36:01.338413 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.337975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43581 (local address 127.0.0.1:40748)
0618 22:36:01.338148 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:01.338245 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260618 22:36:01.387678 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 1170
I20260618 22:36:01.398862 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.398412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43581 (local address 127.0.0.1:40756)
0618 22:36:01.398605 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:01.398712 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260618 22:36:01.399828 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.399464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43581 (local address 127.0.0.1:40762)
0618 22:36:01.399623 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:01.399693 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260618 22:36:01.401912 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 1301
I20260618 22:36:01.414553 1442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.402713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47172 (local address 127.18.150.131:34617)
0618 22:36:01.402898 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:01.402903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:01.402923 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:36:01.403044 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:01.403049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:01.403112 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:01.403191 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:01.404425 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.405178 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.406467 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:01.406865 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:01.406972 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:01.409308 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:01.409323 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:01.409339 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:01.409375 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:01.411866 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:01.412632 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:01.412640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:01.412642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:01.412769 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:01.413115 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:01.413119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:01.413121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:01.413459 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:01.413739 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:01.414029 (+ 290us) server_negotiation.cc:299] Negotiation successful
0618 22:36:01.414258 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:36:01.415191 1424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:01.480981 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.480421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43581 (local address 127.0.0.1:40778)
0618 22:36:01.480689 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:01.480802 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260618 22:36:01.482180 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:01.481622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:43581 (local address 127.0.0.1:40784)
0618 22:36:01.481824 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:01.481915 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:43581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260618 22:36:01.502920 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 1301
I20260618 22:36:01.516734 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 948
I20260618 22:36:01.517823 1011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:01.628667 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 948
I20260618 22:36:01.645903 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39461
--webserver_interface=127.18.150.190
--webserver_port=42805
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:39461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:36:01.962177 1479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:01.962471 1479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:01.962597 1479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:01.969905 1479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:36:01.970052 1479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:01.970129 1479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:01.970206 1479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:36:01.970266 1479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:36:01.979967 1479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:39461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=42805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:01.981251 1479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:01.982513 1479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:01.993515 1485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:01.993518 1487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:01.993570 1484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:01.994433 1479 server_base.cc:1034] running on GCE node
I20260618 22:36:01.995157 1479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:01.996465 1479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:01.997660 1479 hybrid_clock.cc:648] HybridClock initialized: now 1781822161997618 us; error 28 us; skew 500 ppm
I20260618 22:36:01.998009 1479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:01 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822161, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:02.000917 1479 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:36:02.002333 1479 webserver.cc:460] Webserver started at http://127.18.150.190:42805/ using document root <none> and password file <none>
I20260618 22:36:02.002956 1479 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:02.003068 1479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:02.008114 1479 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260618 22:36:02.010823 1494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:02.012033 1479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:36:02.012172 1479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "c7c524f35d3d4eca8221fd29b792a6b9"
format_stamp: "Formatted at 2026-06-18 22:35:56 on dist-test-slave-h7jz"
server_key: "b70daf49baa51442803614b38d2184cd"
server_key_iv: "60a16dd8d86cc54c00a320846d0f9cc1"
server_key_version: "encryptionkey@0"
I20260618 22:36:02.013087 1479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:02.040990 1479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:02.044746 1479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:02.044986 1479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:02.061784 1479 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:39461
I20260618 22:36:02.061803 1545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:39461 every 8 connection(s)
I20260618 22:36:02.063189 1479 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:36:02.065544 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 1479
I20260618 22:36:02.066515 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:43581
--local_ip_for_outbound_sockets=127.18.150.129
--tserver_master_addrs=127.18.150.190:39461
--webserver_port=43483
--webserver_interface=127.18.150.129
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:36:02.071890 1546 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:36:02.075986 1546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Bootstrap starting.
I20260618 22:36:02.095821 1546 log.cc:824] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:02.111498 1546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:36:02.112004 1546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9: Bootstrap complete.
I20260618 22:36:02.121240 1546 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:36:02.122154 1546 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7c524f35d3d4eca8221fd29b792a6b9, State: Initialized, Role: FOLLOWER
I20260618 22:36:02.122750 1546 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:36:02.122972 1546 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:36:02.123080 1546 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:36:02.123238 1546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 2 FOLLOWER]: Advancing to term 3
I20260618 22:36:02.126960 1546 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:36:02.127421 1546 leader_election.cc:302] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7c524f35d3d4eca8221fd29b792a6b9; no voters:
I20260618 22:36:02.128098 1546 leader_election.cc:288] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260618 22:36:02.128243 1550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 3 FOLLOWER]: Leader election won for term 3
I20260618 22:36:02.129284 1550 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [term 3 LEADER]: Becoming Leader. State: Replica: c7c524f35d3d4eca8221fd29b792a6b9, State: Running, Role: LEADER
I20260618 22:36:02.129917 1550 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } }
I20260618 22:36:02.130237 1546 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:36:02.132485 1552 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7c524f35d3d4eca8221fd29b792a6b9. Latest consensus state: current_term: 3 leader_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } } }
I20260618 22:36:02.132916 1552 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:36:02.132491 1551 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c524f35d3d4eca8221fd29b792a6b9" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39461 } } }
I20260618 22:36:02.133239 1551 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7c524f35d3d4eca8221fd29b792a6b9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:36:02.139516 1558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:36:02.148823 1558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:36:02.150597 1558 catalog_manager.cc:1259] Loaded cluster ID: 6cda1eaba9f64238a81e7dec29335c02
I20260618 22:36:02.150707 1558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260618 22:36:02.152807 1558 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5a6be679ad google::LogMessage::Fail() at ??:0
@ 0x7f5a6be6a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f5a6be674de google::LogMessage::Flush() at ??:0
@ 0x7f5a6be68a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5a8189a512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f5a81897976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5a8191f610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
I20260618 22:36:02.177335 1568 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f5a8191f439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5a6eab3345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5a6eab691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f5a6eab6799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5a6ea91eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5a71d6b6db start_thread at ??:0
@ 0x7f5a6ad0161f clone at ??:0
W20260618 22:36:02.415716 1548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:02.415989 1548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:02.416112 1548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:02.423385 1548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:02.423508 1548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:02.423656 1548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:36:02.433307 1548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:43581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=43483
--tserver_master_addrs=127.18.150.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:02.434643 1548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:02.435941 1548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:02.448550 1587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:02.448582 1590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:02.448513 1588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:02.449132 1548 server_base.cc:1034] running on GCE node
I20260618 22:36:02.449811 1548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:02.450800 1548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:02.451998 1548 hybrid_clock.cc:648] HybridClock initialized: now 1781822162451952 us; error 35 us; skew 500 ppm
I20260618 22:36:02.452297 1548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:02 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822162, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:02.455384 1548 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:36:02.456749 1548 webserver.cc:460] Webserver started at http://127.18.150.129:43483/ using document root <none> and password file <none>
I20260618 22:36:02.457406 1548 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:02.457531 1548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:02.462723 1548 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:36:02.465590 1597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:02.466930 1548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:36:02.467073 1548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "ceeffefe2d6f4430971feaeef9474273"
format_stamp: "Formatted at 2026-06-18 22:35:57 on dist-test-slave-h7jz"
server_key: "8a745f92ca8f614560a142606547c688"
server_key_iv: "a11056c022482671461af1a918768374"
server_key_version: "encryptionkey@0"
I20260618 22:36:02.467991 1548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:02.487880 1548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:02.491600 1548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:02.491834 1548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:02.492733 1548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:36:02.494397 1604 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260618 22:36:02.499739 1548 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260618 22:36:02.499840 1548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260618 22:36:02.499954 1548 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260618 22:36:02.502962 1548 ts_tablet_manager.cc:610] Registered 1 tablets
I20260618 22:36:02.503036 1548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260618 22:36:02.503348 1604 tablet_bootstrap.cc:492] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Bootstrap starting.
I20260618 22:36:02.528843 1604 log.cc:824] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:02.536003 1548 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:43581
I20260618 22:36:02.536041 1711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:43581 every 8 connection(s)
I20260618 22:36:02.537482 1548 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:36:02.540716 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 1548
I20260618 22:36:02.541535 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:46345
--local_ip_for_outbound_sockets=127.18.150.130
--tserver_master_addrs=127.18.150.190:39461
--webserver_port=36495
--webserver_interface=127.18.150.130
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:36:02.542814 1604 tablet_bootstrap.cc:492] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:36:02.543398 1604 tablet_bootstrap.cc:492] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Bootstrap complete.
I20260618 22:36:02.544143 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:02.540412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:42657)
0618 22:36:02.542527 (+ 2115us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:02.542972 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1810,"thread_start_us":1545,"threads_started":1}
W20260618 22:36:02.544904 1712 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:39461 (0 consecutive failures): Network error: Failed to ping master at 127.18.150.190:39461: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
I20260618 22:36:02.544955 1604 ts_tablet_manager.cc:1397] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Time spent bootstrapping tablet: real 0.042s user 0.036s sys 0.005s
I20260618 22:36:02.545882 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:02.545505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:55081)
0618 22:36:02.545668 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:02.545753 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260618 22:36:02.546911 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:02.546503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:57745)
0618 22:36:02.546658 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:02.546754 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260618 22:36:02.547194 1712 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260618 22:36:02.551851 1604 raft_consensus.cc:348] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:02.552695 1604 raft_consensus.cc:729] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceeffefe2d6f4430971feaeef9474273, State: Initialized, Role: FOLLOWER
I20260618 22:36:02.553265 1604 consensus_queue.cc:260] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:02.554749 1604 ts_tablet_manager.cc:1428] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260618 22:36:02.554886 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:02.554475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:33079)
0618 22:36:02.554633 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:02.554720 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260618 22:36:02.555724 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:02.555396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:34803)
0618 22:36:02.555530 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:02.555599 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260618 22:36:02.857235 1715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:02.857513 1715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:02.857649 1715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:02.864841 1715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:02.864954 1715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:02.865103 1715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:36:02.874825 1715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:46345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=36495
--tserver_master_addrs=127.18.150.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:02.876042 1715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:02.877326 1715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:02.889760 1724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:02.889707 1721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:02.889732 1722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:02.890292 1715 server_base.cc:1034] running on GCE node
I20260618 22:36:02.890921 1715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:02.891810 1715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:02.893026 1715 hybrid_clock.cc:648] HybridClock initialized: now 1781822162892973 us; error 69 us; skew 500 ppm
I20260618 22:36:02.893337 1715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:02 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822162, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:02.896340 1715 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:36:02.897743 1715 webserver.cc:460] Webserver started at http://127.18.150.130:36495/ using document root <none> and password file <none>
I20260618 22:36:02.898456 1715 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:02.898572 1715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:02.903867 1715 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260618 22:36:02.906618 1731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:02.908090 1715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:36:02.908283 1715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "18e7225647754aaa966d806975f09b11"
format_stamp: "Formatted at 2026-06-18 22:35:57 on dist-test-slave-h7jz"
server_key: "0c42ac44865880320234b4e0c97031e4"
server_key_iv: "c133ac88af74ba5d276c4b488cdff546"
server_key_version: "encryptionkey@0"
I20260618 22:36:02.909312 1715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:02.935477 1715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:02.939677 1715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:02.939949 1715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:02.941030 1715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:36:02.942487 1715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:36:02.942596 1715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:02.942710 1715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:36:02.942787 1715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:02.974148 1715 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:46345
I20260618 22:36:02.974180 1843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:46345 every 8 connection(s)
I20260618 22:36:02.975666 1715 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:36:02.980402 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:02.978573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:34283)
0618 22:36:02.979301 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:02.979529 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":275,"threads_started":1}
W20260618 22:36:02.981058 1844 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:39461 (0 consecutive failures): Network error: Failed to ping master at 127.18.150.190:39461: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
I20260618 22:36:02.981927 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:02.981556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:56645)
0618 22:36:02.981729 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:02.981808 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260618 22:36:02.982786 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:02.982468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:58607)
0618 22:36:02.982593 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:02.982676 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260618 22:36:02.983021 1844 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260618 22:36:02.984650 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 1715
I20260618 22:36:02.985308 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:34617
--local_ip_for_outbound_sockets=127.18.150.131
--tserver_master_addrs=127.18.150.190:39461
--webserver_port=34111
--webserver_interface=127.18.150.131
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:36:03.292876 1847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:03.293138 1847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:03.293267 1847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:03.300601 1847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:03.300709 1847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:03.300856 1847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:36:03.310503 1847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:34617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=34111
--tserver_master_addrs=127.18.150.190:39461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:03.311749 1847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:03.313074 1847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:03.325349 1852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:03.325314 1853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:03.325497 1855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:03.326259 1847 server_base.cc:1034] running on GCE node
I20260618 22:36:03.326846 1847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:03.327726 1847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:03.328922 1847 hybrid_clock.cc:648] HybridClock initialized: now 1781822163328865 us; error 51 us; skew 500 ppm
I20260618 22:36:03.329212 1847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:03 dist-test-slave-h7jz krb5kdc[397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822163, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:03.332134 1847 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:36:03.333496 1847 webserver.cc:460] Webserver started at http://127.18.150.131:34111/ using document root <none> and password file <none>
I20260618 22:36:03.334224 1847 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:03.334321 1847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:03.339232 1847 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260618 22:36:03.341748 1862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:03.343011 1847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:36:03.343137 1847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "5d75adde76af449f953f0dfb3a9bcf5c"
format_stamp: "Formatted at 2026-06-18 22:35:58 on dist-test-slave-h7jz"
server_key: "a11ab6c802f4c1c8d3dc0c75b31b8b2f"
server_key_iv: "b73311f52a8d439800f3e0ec2b845849"
server_key_version: "encryptionkey@0"
I20260618 22:36:03.344007 1847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:03.374650 1847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:03.378269 1847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:03.378484 1847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:03.379334 1847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:36:03.380570 1847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:36:03.380676 1847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:03.380746 1847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:36:03.380801 1847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:03.409489 1847 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:34617
I20260618 22:36:03.409557 1974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:34617 every 8 connection(s)
I20260618 22:36:03.410916 1847 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:36:03.414848 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 1847
I20260618 22:36:03.415952 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:03.413799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:42065)
0618 22:36:03.414604 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:03.414883 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":282,"threads_started":1}
W20260618 22:36:03.416702 1975 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:39461 (0 consecutive failures): Network error: Failed to ping master at 127.18.150.190:39461: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
I20260618 22:36:03.417434 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:03.417142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:41159)
0618 22:36:03.417264 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:03.417328 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260618 22:36:03.418226 1977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:03.417907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:41747)
0618 22:36:03.418047 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:03.418124 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260618 22:36:03.418437 1975 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260618 22:36:03.557451 1979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:03.556526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:52191)
0618 22:36:03.557028 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:03.557126 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":229,"threads_started":1}
I20260618 22:36:03.984337 1980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:03.983578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:51865)
0618 22:36:03.984009 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:03.984125 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":199,"threads_started":1}
I20260618 22:36:04.150995 1981 raft_consensus.cc:482] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:36:04.151309 1981 raft_consensus.cc:504] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:04.152805 1981 leader_election.cc:288] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617), 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345)
Jun 18 22:36:04 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822162, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
Jun 18 22:36:04 dist-test-slave-h7jz krb5kdc[397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822162, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:36:04.178036 1983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:04.153265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:34541 (local address 127.18.150.131:34617)
0618 22:36:04.154033 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:04.154049 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:04.154098 (+ 49us) server_negotiation.cc:407] Connection header received
0618 22:36:04.155870 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:04.155878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:04.156243 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:04.156516 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:04.158632 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:04.159780 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:04.161471 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:04.162183 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:04.162355 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:04.171251 (+ 8896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:04.171261 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:04.171279 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:04.171320 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:04.174464 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:04.175304 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:04.175311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:04.175316 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:04.175475 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:04.175883 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:04.175886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:04.175888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:04.176387 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:04.176552 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:04.177295 (+ 743us) server_negotiation.cc:299] Negotiation successful
0618 22:36:04.177618 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":449,"threads_started":1}
I20260618 22:36:04.178061 1984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:04.153799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:46345 (local address 127.18.150.129:47315)
0618 22:36:04.154313 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:04.154328 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:04.154426 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:04.157217 (+ 2791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:04.157221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:04.157761 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:36:04.158444 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:04.158465 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:04.159934 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:04.159938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:04.161340 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:04.161352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:04.162660 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:04.162665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:04.162798 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:04.168238 (+ 5440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:36:04.168270 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:36:04.171145 (+ 2875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:36:04.174623 (+ 3478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:04.174629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:04.174638 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:04.175170 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:04.175567 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:04.175573 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:04.175577 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:04.175770 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:04.176503 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:36:04.176510 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:36:04.176815 (+ 305us) client_negotiation.cc:769] Sending connection context
0618 22:36:04.177049 (+ 234us) client_negotiation.cc:240] Negotiation successful
0618 22:36:04.177579 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":161,"spinlock_wait_cycles":77056,"thread_start_us":595,"threads_started":1}
I20260618 22:36:04.178056 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:04.154283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:47315 (local address 127.18.150.130:46345)
0618 22:36:04.154961 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:04.154972 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:04.155005 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:36:04.156452 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:04.156462 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:04.156899 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:04.157230 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:04.158625 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:04.159745 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:04.161472 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:04.162240 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:04.162401 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:04.171401 (+ 9000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:04.171409 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:04.171422 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:04.171465 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:04.174497 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:04.175301 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:04.175308 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:04.175314 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:04.175460 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:04.175874 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:04.175878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:04.175880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:04.176375 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:04.176541 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:04.177287 (+ 746us) server_negotiation.cc:299] Negotiation successful
0618 22:36:04.177643 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":389,"threads_started":1}
I20260618 22:36:04.178146 1982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:04.153097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:34617 (local address 127.18.150.129:34541)
0618 22:36:04.153606 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:04.153641 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:04.154222 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:04.156708 (+ 2486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:04.156718 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:04.157719 (+ 1001us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:36:04.158444 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:04.158464 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:04.159940 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:04.159944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:04.161334 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:04.161346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:04.162373 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:04.162380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:04.162765 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:04.168238 (+ 5473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:36:04.168267 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:36:04.170891 (+ 2624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:36:04.174612 (+ 3721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:04.174621 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:04.174637 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:04.175167 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:04.175578 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:04.175583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:04.175587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:04.175777 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:04.176503 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:36:04.176513 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:36:04.176816 (+ 303us) client_negotiation.cc:769] Sending connection context
0618 22:36:04.177048 (+ 232us) client_negotiation.cc:240] Negotiation successful
0618 22:36:04.177710 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":344,"spinlock_wait_cycles":19456,"thread_start_us":254,"threads_started":1}
W20260618 22:36:04.181044 1598 leader_election.cc:341] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345): Not found: Tablet not found: 4f76619e34094466a28e7b77d79aadff
W20260618 22:36:04.181331 1598 leader_election.cc:341] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617): Not found: Tablet not found: 4f76619e34094466a28e7b77d79aadff
I20260618 22:36:04.181442 1598 leader_election.cc:302] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceeffefe2d6f4430971feaeef9474273; no voters: 18e7225647754aaa966d806975f09b11, 5d75adde76af449f953f0dfb3a9bcf5c
I20260618 22:36:04.181865 1981 raft_consensus.cc:2726] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:36:04.419687 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:04.418893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:53255)
0618 22:36:04.419307 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:04.419436 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":189,"threads_started":1}
I20260618 22:36:04.558799 1984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:04.558354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:60937)
0618 22:36:04.558545 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:04.558657 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260618 22:36:04.985836 1987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:04.985187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:33841)
0618 22:36:04.985604 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:04.985686 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":216,"threads_started":1}
I20260618 22:36:05.421272 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:05.420577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:51249)
0618 22:36:05.421021 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:05.421101 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":262,"threads_started":1}
I20260618 22:36:05.560410 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:05.559647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:55669)
0618 22:36:05.560130 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:05.560232 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":212,"threads_started":1}
I20260618 22:36:05.987349 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:05.986678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:46835)
0618 22:36:05.987096 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:05.987180 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":201,"threads_started":1}
I20260618 22:36:06.232559 1991 raft_consensus.cc:482] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:36:06.232774 1991 raft_consensus.cc:504] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:06.233537 1991 leader_election.cc:288] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617), 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345)
W20260618 22:36:06.234561 1598 leader_election.cc:341] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617): Not found: Tablet not found: 4f76619e34094466a28e7b77d79aadff
W20260618 22:36:06.234797 1598 leader_election.cc:341] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345): Not found: Tablet not found: 4f76619e34094466a28e7b77d79aadff
I20260618 22:36:06.234934 1598 leader_election.cc:302] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceeffefe2d6f4430971feaeef9474273; no voters: 18e7225647754aaa966d806975f09b11, 5d75adde76af449f953f0dfb3a9bcf5c
I20260618 22:36:06.235240 1991 raft_consensus.cc:2726] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:36:06.422693 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:06.422061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:38609)
0618 22:36:06.422467 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:06.422544 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":199,"threads_started":1}
I20260618 22:36:06.562037 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:06.561247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:42581)
0618 22:36:06.561679 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:06.561806 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":215,"threads_started":1}
I20260618 22:36:06.988778 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:06.988110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:47629)
0618 22:36:06.988526 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:06.988606 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":201,"threads_started":1}
I20260618 22:36:07.424226 1995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:07.423419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:49963)
0618 22:36:07.423961 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:07.424044 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":277,"threads_started":1}
I20260618 22:36:07.563705 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:07.563021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:45721)
0618 22:36:07.563429 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:07.563528 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":205,"threads_started":1}
I20260618 22:36:07.990391 1997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:07.989635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:40065)
0618 22:36:07.990138 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:07.990222 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":246,"threads_started":1}
I20260618 22:36:08.378357 1998 raft_consensus.cc:482] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:36:08.378547 1998 raft_consensus.cc:504] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:08.379128 1998 leader_election.cc:288] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617), 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345)
W20260618 22:36:08.380059 1598 leader_election.cc:341] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617): Not found: Tablet not found: 4f76619e34094466a28e7b77d79aadff
W20260618 22:36:08.380246 1598 leader_election.cc:341] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345): Not found: Tablet not found: 4f76619e34094466a28e7b77d79aadff
I20260618 22:36:08.380359 1598 leader_election.cc:302] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceeffefe2d6f4430971feaeef9474273; no voters: 18e7225647754aaa966d806975f09b11, 5d75adde76af449f953f0dfb3a9bcf5c
I20260618 22:36:08.380595 1998 raft_consensus.cc:2726] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:36:08.425808 1999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:08.425093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:47743)
0618 22:36:08.425555 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:08.425642 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":239,"threads_started":1}
I20260618 22:36:08.565202 2000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:08.564486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:34409)
0618 22:36:08.564920 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:08.565008 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":234,"threads_started":1}
I20260618 22:36:08.992033 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:08.991295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:37811)
0618 22:36:08.991747 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:08.991841 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":232,"threads_started":1}
I20260618 22:36:09.427341 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:09.426662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:41975)
0618 22:36:09.427092 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:09.427173 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":225,"threads_started":1}
I20260618 22:36:09.566880 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:09.566141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:33803)
0618 22:36:09.566592 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:09.566692 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":238,"threads_started":1}
I20260618 22:36:09.993932 2004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:09.993071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:47033)
0618 22:36:09.993651 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:09.993753 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":254,"threads_started":1}
I20260618 22:36:10.429092 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:10.428280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:39725)
0618 22:36:10.428816 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:10.428913 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":268,"threads_started":1}
I20260618 22:36:10.568586 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:10.567823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:46875)
0618 22:36:10.568322 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:10.568417 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":252,"threads_started":1}
I20260618 22:36:10.995800 2007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:10.995039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:45371)
0618 22:36:10.995533 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:10.995627 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":245,"threads_started":1}
I20260618 22:36:11.430794 2008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:11.430080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:60001)
0618 22:36:11.430509 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:11.430615 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":213,"threads_started":1}
I20260618 22:36:11.570405 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:11.569583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:34959)
0618 22:36:11.570127 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:11.570224 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":248,"threads_started":1}
I20260618 22:36:11.997332 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:11.996664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:59083)
0618 22:36:11.997089 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:11.997177 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":199,"threads_started":1}
I20260618 22:36:12.432364 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:12.431668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:41077)
0618 22:36:12.432131 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:12.432210 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":199,"threads_started":1}
I20260618 22:36:12.572098 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:12.571410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:47969)
0618 22:36:12.571820 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:12.571904 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":199,"threads_started":1}
I20260618 22:36:12.831604 2013 raft_consensus.cc:482] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:36:12.831835 2013 raft_consensus.cc:504] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:12.832444 2013 leader_election.cc:288] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617), 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345)
W20260618 22:36:12.833289 1598 leader_election.cc:341] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617): Not found: Tablet not found: 4f76619e34094466a28e7b77d79aadff
W20260618 22:36:12.833489 1598 leader_election.cc:341] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345): Not found: Tablet not found: 4f76619e34094466a28e7b77d79aadff
I20260618 22:36:12.833570 1598 leader_election.cc:302] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceeffefe2d6f4430971feaeef9474273; no voters: 18e7225647754aaa966d806975f09b11, 5d75adde76af449f953f0dfb3a9bcf5c
I20260618 22:36:12.833767 2013 raft_consensus.cc:2726] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:36:12.998925 2014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:12.998218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:36685)
0618 22:36:12.998667 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:12.998750 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":237,"threads_started":1}
I20260618 22:36:13.433926 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:13.433222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:57167)
0618 22:36:13.433654 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:13.433738 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":215,"threads_started":1}
I20260618 22:36:13.573714 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:13.572950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:50899)
0618 22:36:13.573454 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:13.573544 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":240,"threads_started":1}
I20260618 22:36:14.000607 2017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:13.999863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:40141)
0618 22:36:14.000353 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:14.000438 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":237,"threads_started":1}
I20260618 22:36:14.435385 2018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:14.434709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:38281)
0618 22:36:14.435149 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:14.435239 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":224,"threads_started":1}
I20260618 22:36:14.575331 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:14.574603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:47329)
0618 22:36:14.575046 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:14.575145 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":218,"threads_started":1}
I20260618 22:36:14.733171 2020 raft_consensus.cc:482] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:36:14.733352 2020 raft_consensus.cc:504] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceeffefe2d6f4430971feaeef9474273" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 43581 } } peers { permanent_uuid: "5d75adde76af449f953f0dfb3a9bcf5c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 34617 } } peers { permanent_uuid: "18e7225647754aaa966d806975f09b11" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 46345 } }
I20260618 22:36:14.733920 2020 leader_election.cc:288] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617), 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345)
W20260618 22:36:14.734854 1598 leader_election.cc:341] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d75adde76af449f953f0dfb3a9bcf5c (127.18.150.131:34617): Not found: Tablet not found: 4f76619e34094466a28e7b77d79aadff
W20260618 22:36:14.735075 1598 leader_election.cc:341] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 18e7225647754aaa966d806975f09b11 (127.18.150.130:46345): Not found: Tablet not found: 4f76619e34094466a28e7b77d79aadff
I20260618 22:36:14.735183 1598 leader_election.cc:302] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ceeffefe2d6f4430971feaeef9474273; no voters: 18e7225647754aaa966d806975f09b11, 5d75adde76af449f953f0dfb3a9bcf5c
I20260618 22:36:14.735438 2020 raft_consensus.cc:2726] T 4f76619e34094466a28e7b77d79aadff P ceeffefe2d6f4430971feaeef9474273 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:36:15.002147 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:15.001430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:58755)
0618 22:36:15.001846 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:15.001924 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":214,"threads_started":1}
I20260618 22:36:15.437040 2022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:15.436270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:44229)
0618 22:36:15.436745 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:15.436830 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":258,"threads_started":1}
I20260618 22:36:15.576879 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:15.576188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:48645)
0618 22:36:15.576603 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:15.576688 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":215,"threads_started":1}
I20260618 22:36:16.003690 2024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:16.002923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:58753)
0618 22:36:16.003394 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:16.003499 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":247,"threads_started":1}
I20260618 22:36:16.438516 2025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:16.437788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:49507)
0618 22:36:16.438255 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:16.438348 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":263,"threads_started":1}
I20260618 22:36:16.578540 2026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:16.577739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:50427)
0618 22:36:16.578270 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:16.578359 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":267,"threads_started":1}
I20260618 22:36:17.005318 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:17.004579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:52635)
0618 22:36:17.005030 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:17.005125 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":225,"threads_started":1}
I20260618 22:36:17.440069 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:17.439324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:41293)
0618 22:36:17.439775 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:17.439859 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":231,"threads_started":1}
I20260618 22:36:17.580595 2029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:17.579800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.129:49489)
0618 22:36:17.580285 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:17.580373 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":254,"threads_started":1}
I20260618 22:36:18.007189 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:18.006446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.130:44005)
0618 22:36:18.006904 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:18.007012 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":222,"threads_started":1}
I20260618 22:36:18.417402 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 1548
I20260618 22:36:18.432476 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:18.418623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35632 (local address 127.18.150.129:43581)
0618 22:36:18.419197 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:18.419207 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:18.419240 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:36:18.419314 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:18.419321 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:18.419552 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:18.419715 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:18.421537 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:18.422664 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:18.423905 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:18.424439 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:18.424552 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:18.427185 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:18.427197 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:18.427212 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:18.427244 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:18.429818 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:18.430463 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:18.430470 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:18.430472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:18.430600 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:18.430970 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:18.430975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:18.430979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:18.431317 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:18.431615 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:18.431915 (+ 300us) server_negotiation.cc:299] Negotiation successful
0618 22:36:18.432167 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":244,"threads_started":1}
I20260618 22:36:18.433688 1707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:18.442055 2033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:18.441219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39461 (local address 127.18.150.131:52283)
0618 22:36:18.441706 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:18.441799 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":252,"threads_started":1}
I20260618 22:36:18.522212 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 1548
I20260618 22:36:18.537003 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 1715
I20260618 22:36:18.549919 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:18.537779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41376 (local address 127.18.150.130:46345)
0618 22:36:18.538393 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:18.538397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:18.538426 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:36:18.538483 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:18.538486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:18.538545 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:18.538634 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:18.540014 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:18.540804 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:18.542036 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:18.542461 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:18.542553 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:18.544897 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:18.544901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:18.544905 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:18.544938 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:18.547558 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:18.548237 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:18.548241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:18.548243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:18.548379 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:18.548736 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:18.548740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:18.548742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:18.549006 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:18.549208 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:18.549407 (+ 199us) server_negotiation.cc:299] Negotiation successful
0618 22:36:18.549579 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":396,"threads_started":1}
I20260618 22:36:18.550709 1839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:18.632740 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 1715
I20260618 22:36:18.647620 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 1847
I20260618 22:36:18.660666 2036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:18.648424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51042 (local address 127.18.150.131:34617)
0618 22:36:18.648957 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:18.648961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:18.648979 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:36:18.649032 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:18.649036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:18.649095 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:18.649182 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:18.650666 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:18.651447 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:18.652663 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:18.653064 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:18.653152 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:18.655853 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:18.655857 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:18.655861 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:18.655893 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:18.658352 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:18.658981 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:18.658985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:18.658987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:18.659129 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:18.659512 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:18.659515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:18.659517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:18.659748 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:18.659933 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:18.660213 (+ 280us) server_negotiation.cc:299] Negotiation successful
0618 22:36:18.660365 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":275,"threads_started":1}
I20260618 22:36:18.661273 1970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:18.745003 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 1847
2026-06-18T22:36:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (23792 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:36:18 dist-test-slave-h7jz krb5kdc[2040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:36:18 dist-test-slave-h7jz krb5kdc[2040](info): set up 2 sockets
Jun 18 22:36:18 dist-test-slave-h7jz krb5kdc[2040](info): commencing operation
krb5kdc: starting...
W20260618 22:36:19.754583 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:36:19 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:36:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:36:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:36:19.971879 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:46483
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:46483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:36:20.286127 2053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:20.286409 2053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:20.286543 2053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:20.293538 2053 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260618 22:36:20.293638 2053 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260618 22:36:20.293690 2053 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260618 22:36:20.293758 2053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:36:20.293833 2053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:20.293931 2053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:20.294005 2053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:36:20.294080 2053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:36:20.303753 2053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:46483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:46483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:20.305002 2053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:20.306306 2053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:20.317817 2058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:20.317792 2061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:20.317884 2059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:20.318459 2053 server_base.cc:1034] running on GCE node
I20260618 22:36:20.319252 2053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:20.320641 2053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:20.321890 2053 hybrid_clock.cc:648] HybridClock initialized: now 1781822180321824 us; error 34 us; skew 500 ppm
I20260618 22:36:20.322238 2053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:20 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822180, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:20.325166 2053 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:36:20.326517 2053 webserver.cc:460] Webserver started at http://127.18.150.190:46055/ using document root <none> and password file <none>
I20260618 22:36:20.327150 2053 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:20.327231 2053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:20.327518 2053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:36:20.330413 2053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "c70be62673614a6ea5293395381c1f55"
format_stamp: "Formatted at 2026-06-18 22:36:20 on dist-test-slave-h7jz"
server_key: "7a41200c303f1de255d1aff511d659ef"
server_key_iv: "86d8dadc71b30d35fa9137f892ac0af4"
server_key_version: "encryptionkey@0"
I20260618 22:36:20.331115 2053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "c70be62673614a6ea5293395381c1f55"
format_stamp: "Formatted at 2026-06-18 22:36:20 on dist-test-slave-h7jz"
server_key: "7a41200c303f1de255d1aff511d659ef"
server_key_iv: "86d8dadc71b30d35fa9137f892ac0af4"
server_key_version: "encryptionkey@0"
I20260618 22:36:20.336086 2053 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:36:20.339359 2068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:20.340720 2053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:36:20.340857 2053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "c70be62673614a6ea5293395381c1f55"
format_stamp: "Formatted at 2026-06-18 22:36:20 on dist-test-slave-h7jz"
server_key: "7a41200c303f1de255d1aff511d659ef"
server_key_iv: "86d8dadc71b30d35fa9137f892ac0af4"
server_key_version: "encryptionkey@0"
I20260618 22:36:20.341037 2053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:20.353874 2053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:20.357630 2053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:20.357833 2053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:20.374307 2053 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:46483
I20260618 22:36:20.374326 2119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:46483 every 8 connection(s)
I20260618 22:36:20.375562 2053 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:36:20.379015 2120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:36:20.382767 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 2053
I20260618 22:36:20.382995 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:36:20.383787 19034 external_mini_cluster.cc:1428] Setting key 506b0a261a1537c87ffb85df3bfc73c5
I20260618 22:36:20.391870 2120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Bootstrap starting.
I20260618 22:36:20.395416 2120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Neither blocks nor log segments found. Creating new log.
I20260618 22:36:20.396651 2120 log.cc:824] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:36:20 dist-test-slave-h7jz krb5kdc[2040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:36:20.399652 2120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: No bootstrap required, opened a new log
I20260618 22:36:20.407027 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:20.385706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38130 (local address 127.18.150.190:46483)
0618 22:36:20.386424 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:20.386435 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:20.386474 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:36:20.387996 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:20.388017 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:20.388417 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:20.388746 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:20.390523 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:20.391803 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:20.393640 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:20.394326 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:20.394450 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:20.399154 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:20.399162 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:20.399176 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:20.399213 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:20.402331 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:20.403258 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:20.403269 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:20.403277 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:20.403499 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:20.403987 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:20.403992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:20.403995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:20.404795 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:20.405088 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:20.405592 (+ 504us) server_negotiation.cc:299] Negotiation successful
0618 22:36:20.405973 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":336,"threads_started":1}
I20260618 22:36:20.408447 2120 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:36:20.408787 2120 raft_consensus.cc:374] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:36:20.408877 2120 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70be62673614a6ea5293395381c1f55, State: Initialized, Role: FOLLOWER
I20260618 22:36:20.409408 2120 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:36:20.409641 2120 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:36:20.409766 2120 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:36:20.409904 2120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:36:20.411979 2120 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:36:20.412438 2120 leader_election.cc:302] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70be62673614a6ea5293395381c1f55; no voters:
I20260618 22:36:20.412953 2120 leader_election.cc:288] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:36:20.413071 2125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:36:20.413666 2125 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 1 LEADER]: Becoming Leader. State: Replica: c70be62673614a6ea5293395381c1f55, State: Running, Role: LEADER
I20260618 22:36:20.414248 2125 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:36:20.414891 2120 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:36:20.416623 2126 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c70be62673614a6ea5293395381c1f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } } }
I20260618 22:36:20.416954 2126 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: This master's current role is: LEADER
I20260618 22:36:20.418159 2127 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70be62673614a6ea5293395381c1f55. Latest consensus state: current_term: 1 leader_uuid: "c70be62673614a6ea5293395381c1f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } } }
I20260618 22:36:20.418558 2127 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: This master's current role is: LEADER
I20260618 22:36:20.421900 2130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:36:20.426791 2130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:36:20.434579 2130 catalog_manager.cc:1347] Generated new cluster ID: 18df41e3270940f1a382ae3e02efea5d
I20260618 22:36:20.434676 2130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:36:20.448544 2130 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:36:20.449812 2130 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:36:20.457751 2130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Generated new TSK 0
I20260618 22:36:20.458544 2130 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:36:20.513713 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46483
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:36:20.836211 2146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:20.836544 2146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:20.836702 2146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:20.843997 2146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:20.844103 2146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:20.844223 2146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:36:20.853854 2146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:20.855047 2146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:20.856267 2146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:20.868978 2154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:20.868918 2152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:20.869016 2151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:20.869853 2146 server_base.cc:1034] running on GCE node
I20260618 22:36:20.870471 2146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:20.871390 2146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:20.872591 2146 hybrid_clock.cc:648] HybridClock initialized: now 1781822180872538 us; error 45 us; skew 500 ppm
I20260618 22:36:20.872886 2146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:20 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822180, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:20.875839 2146 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:36:20.877251 2146 webserver.cc:460] Webserver started at http://127.18.150.129:36369/ using document root <none> and password file <none>
I20260618 22:36:20.877928 2146 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:20.878067 2146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:20.878319 2146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:36:20.881194 2146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "ea081ab812f14c4dbbc7529e044f30d3"
format_stamp: "Formatted at 2026-06-18 22:36:20 on dist-test-slave-h7jz"
server_key: "db13b6a1ee42a98cf25cca7493c49d93"
server_key_iv: "eaf27c76f04b3c806ff5d131d44e305e"
server_key_version: "encryptionkey@0"
I20260618 22:36:20.882025 2146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "ea081ab812f14c4dbbc7529e044f30d3"
format_stamp: "Formatted at 2026-06-18 22:36:20 on dist-test-slave-h7jz"
server_key: "db13b6a1ee42a98cf25cca7493c49d93"
server_key_iv: "eaf27c76f04b3c806ff5d131d44e305e"
server_key_version: "encryptionkey@0"
I20260618 22:36:20.887183 2146 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260618 22:36:20.890528 2161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:20.891825 2146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:36:20.891956 2146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "ea081ab812f14c4dbbc7529e044f30d3"
format_stamp: "Formatted at 2026-06-18 22:36:20 on dist-test-slave-h7jz"
server_key: "db13b6a1ee42a98cf25cca7493c49d93"
server_key_iv: "eaf27c76f04b3c806ff5d131d44e305e"
server_key_version: "encryptionkey@0"
I20260618 22:36:20.892112 2146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:20.907121 2146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:20.911180 2146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:20.911412 2146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:20.912268 2146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:36:20.913611 2146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:36:20.913682 2146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:20.913771 2146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:36:20.913816 2146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:20.943197 2146 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:38009
I20260618 22:36:20.943229 2273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:38009 every 8 connection(s)
I20260618 22:36:20.944496 2146 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:36:20.948855 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 2146
I20260618 22:36:20.949057 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:36:20.949832 19034 external_mini_cluster.cc:1428] Setting key f1399c8bc46883a6d876e05eb9eeb7b9
Jun 18 22:36:20 dist-test-slave-h7jz krb5kdc[2040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822180, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:36:20.965431 2276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:20.946936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:49729 (local address 127.18.150.190:46483)
0618 22:36:20.947315 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:20.947319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:20.948194 (+ 875us) server_negotiation.cc:407] Connection header received
0618 22:36:20.950613 (+ 2419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:20.950619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:20.950701 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:20.950802 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:20.952520 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:20.953290 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:20.955395 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:20.955831 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:20.955935 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:20.959718 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:20.959725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:20.959733 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:20.959798 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:20.962281 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:20.963111 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:20.963115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:20.963117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:20.963240 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:20.963699 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:20.963702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:20.963704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:20.963989 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:20.964099 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:20.964987 (+ 888us) server_negotiation.cc:299] Negotiation successful
0618 22:36:20.965164 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":191,"threads_started":1}
I20260618 22:36:20.966758 2277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:20.947379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:49729)
0618 22:36:20.948033 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:20.948083 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:20.950305 (+ 2222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:20.951047 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:20.951056 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:20.951578 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:36:20.952322 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:20.952350 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:20.953456 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:20.953462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:20.955239 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:20.955253 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:20.956033 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:20.956040 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:20.956319 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:20.957056 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:36:20.957080 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:36:20.959459 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:36:20.962417 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:20.962425 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:20.962442 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:20.962973 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:20.963342 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:20.963347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:20.963350 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:20.963593 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:20.964102 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:36:20.964110 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:36:20.964549 (+ 439us) client_negotiation.cc:769] Sending connection context
0618 22:36:20.964862 (+ 313us) client_negotiation.cc:240] Negotiation successful
0618 22:36:20.965390 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":250,"threads_started":1}
I20260618 22:36:20.968294 2274 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46483
I20260618 22:36:20.968612 2274 heartbeater.cc:464] Registering TS with master...
I20260618 22:36:20.969523 2274 heartbeater.cc:511] Master 127.18.150.190:46483 requested a full tablet report, sending...
I20260618 22:36:20.971771 2085 ts_manager.cc:194] Registered new tserver with Master: ea081ab812f14c4dbbc7529e044f30d3 (127.18.150.129:38009)
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
I20260618 22:36:20.973217 2085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:49729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:36:21.000788 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46483
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:36:21.311900 2280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:21.312179 2280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:21.312273 2280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:21.319536 2280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:21.319646 2280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:21.319787 2280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:36:21.329401 2280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:21.330602 2280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:21.331820 2280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:21.344408 2285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:21.344372 2288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:21.344338 2286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:21.345098 2280 server_base.cc:1034] running on GCE node
I20260618 22:36:21.345664 2280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:21.346573 2280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:21.347787 2280 hybrid_clock.cc:648] HybridClock initialized: now 1781822181347742 us; error 34 us; skew 500 ppm
I20260618 22:36:21.348078 2280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:21 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822181, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:21.351027 2280 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:36:21.352305 2280 webserver.cc:460] Webserver started at http://127.18.150.130:36421/ using document root <none> and password file <none>
I20260618 22:36:21.352953 2280 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:21.353068 2280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:21.353343 2280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:36:21.356146 2280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "bd6d9ff501c942cca6dae0246ce97c62"
format_stamp: "Formatted at 2026-06-18 22:36:21 on dist-test-slave-h7jz"
server_key: "8c302273fc18059616487156161de372"
server_key_iv: "a96de743e63b45906cea1aec2fe6dfa2"
server_key_version: "encryptionkey@0"
I20260618 22:36:21.356855 2280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "bd6d9ff501c942cca6dae0246ce97c62"
format_stamp: "Formatted at 2026-06-18 22:36:21 on dist-test-slave-h7jz"
server_key: "8c302273fc18059616487156161de372"
server_key_iv: "a96de743e63b45906cea1aec2fe6dfa2"
server_key_version: "encryptionkey@0"
I20260618 22:36:21.361729 2280 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260618 22:36:21.364879 2295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:21.366158 2280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:36:21.366297 2280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "bd6d9ff501c942cca6dae0246ce97c62"
format_stamp: "Formatted at 2026-06-18 22:36:21 on dist-test-slave-h7jz"
server_key: "8c302273fc18059616487156161de372"
server_key_iv: "a96de743e63b45906cea1aec2fe6dfa2"
server_key_version: "encryptionkey@0"
I20260618 22:36:21.366498 2280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:21.380792 2280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:21.385499 2280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:21.385773 2280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:21.386840 2280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:36:21.388329 2280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:36:21.388420 2280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:21.388537 2280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:36:21.388605 2280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:21.419778 2280 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:40131
I20260618 22:36:21.419817 2407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:40131 every 8 connection(s)
I20260618 22:36:21.421065 2280 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:36:21.422123 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 2280
I20260618 22:36:21.422297 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:36:21.422901 19034 external_mini_cluster.cc:1428] Setting key a61a0859d6322fbc3c625b7c3c37c958
Jun 18 22:36:21 dist-test-slave-h7jz krb5kdc[2040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822181, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:36:21.440856 2276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:21.423851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:50135 (local address 127.18.150.190:46483)
0618 22:36:21.424059 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:21.424065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:21.425268 (+ 1203us) server_negotiation.cc:407] Connection header received
0618 22:36:21.427086 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:21.427091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:21.427173 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:21.427258 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:21.428746 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.429460 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.430997 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.431563 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.431695 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:21.435225 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:21.435230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:21.435234 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:21.435266 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:21.437777 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:21.438655 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:21.438659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:21.438662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:21.438796 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:21.439235 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:21.439238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:21.439240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:21.439501 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:21.439623 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:21.440429 (+ 806us) server_negotiation.cc:299] Negotiation successful
0618 22:36:21.440628 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:36:21.442337 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:21.424368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:50135)
0618 22:36:21.425124 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:21.425167 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:21.426830 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:21.427499 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:21.427508 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:21.428021 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:36:21.428583 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:21.428600 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.429601 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.429604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:21.430855 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:21.430867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.431765 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.431770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:21.431981 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:21.432591 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:36:21.432613 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:36:21.435008 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:36:21.437907 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:21.437916 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:21.437933 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:21.438501 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:21.438898 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:21.438903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:21.438906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:21.439136 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:21.439611 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:36:21.439619 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:36:21.440038 (+ 419us) client_negotiation.cc:769] Sending connection context
0618 22:36:21.440347 (+ 309us) client_negotiation.cc:240] Negotiation successful
0618 22:36:21.440901 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":319,"threads_started":1}
I20260618 22:36:21.443832 2408 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46483
I20260618 22:36:21.444181 2408 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:36:21.445003 2408 heartbeater.cc:511] Master 127.18.150.190:46483 requested a full tablet report, sending...
I20260618 22:36:21.446481 2085 ts_manager.cc:194] Registered new tserver with Master: bd6d9ff501c942cca6dae0246ce97c62 (127.18.150.130:40131)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
I20260618 22:36:21.447703 2085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:50135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:36:21.472759 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46483
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:36:21.789299 2413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:21.789567 2413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:21.789686 2413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:21.796841 2413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:21.796958 2413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:21.797113 2413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:36:21.806915 2413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:46483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:21.808218 2413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:21.809518 2413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:21.822234 2419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:21.822283 2418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:21.822302 2421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:21.822918 2413 server_base.cc:1034] running on GCE node
I20260618 22:36:21.823552 2413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:21.824461 2413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:21.825695 2413 hybrid_clock.cc:648] HybridClock initialized: now 1781822181825642 us; error 36 us; skew 500 ppm
I20260618 22:36:21.826081 2413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:21 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822181, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:21.829190 2413 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:36:21.830648 2413 webserver.cc:460] Webserver started at http://127.18.150.131:35203/ using document root <none> and password file <none>
I20260618 22:36:21.831288 2413 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:21.831384 2413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:21.831640 2413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:36:21.834523 2413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "f54f490a53f94619a386da91da12c63c"
format_stamp: "Formatted at 2026-06-18 22:36:21 on dist-test-slave-h7jz"
server_key: "eac2db335a955438425ebfe7bfd18879"
server_key_iv: "cc8c3b2f19f111b6ffa76cafce5f697c"
server_key_version: "encryptionkey@0"
I20260618 22:36:21.835202 2413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "f54f490a53f94619a386da91da12c63c"
format_stamp: "Formatted at 2026-06-18 22:36:21 on dist-test-slave-h7jz"
server_key: "eac2db335a955438425ebfe7bfd18879"
server_key_iv: "cc8c3b2f19f111b6ffa76cafce5f697c"
server_key_version: "encryptionkey@0"
I20260618 22:36:21.840173 2413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260618 22:36:21.843425 2428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:21.844744 2413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260618 22:36:21.844875 2413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "f54f490a53f94619a386da91da12c63c"
format_stamp: "Formatted at 2026-06-18 22:36:21 on dist-test-slave-h7jz"
server_key: "eac2db335a955438425ebfe7bfd18879"
server_key_iv: "cc8c3b2f19f111b6ffa76cafce5f697c"
server_key_version: "encryptionkey@0"
I20260618 22:36:21.845059 2413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:21.861387 2413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:21.865231 2413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:21.865468 2413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:21.866395 2413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:36:21.867630 2413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:36:21.867718 2413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:21.867808 2413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:36:21.867849 2413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:21.896853 2413 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:38409
I20260618 22:36:21.896906 2540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:38409 every 8 connection(s)
I20260618 22:36:21.898279 2413 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:36:21.903565 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 2413
I20260618 22:36:21.903715 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:36:21.904249 19034 external_mini_cluster.cc:1428] Setting key c0e8f11970bf7e12687495cd95fba253
Jun 18 22:36:21 dist-test-slave-h7jz krb5kdc[2040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822181, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:36:21.917032 2276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:21.900821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:51127 (local address 127.18.150.190:46483)
0618 22:36:21.901033 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:21.901040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:21.902216 (+ 1176us) server_negotiation.cc:407] Connection header received
0618 22:36:21.904021 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:21.904028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:21.904114 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:21.904211 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:21.905993 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.906759 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.908313 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.908701 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.908806 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:21.911974 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:21.911978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:21.911981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:21.912013 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:21.914204 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:21.914875 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:21.914880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:21.914884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:21.914993 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:21.915402 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:21.915406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:21.915410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:21.915699 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:21.915843 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:21.916476 (+ 633us) server_negotiation.cc:299] Negotiation successful
0618 22:36:21.916707 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:36:21.918112 2543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:21.901378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:51127)
0618 22:36:21.902075 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:21.902118 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:21.903730 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:21.904431 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:21.904439 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:21.905102 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:36:21.905797 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:21.905815 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.906906 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.906910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:21.908188 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:21.908200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.908906 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.908914 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:21.909148 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:21.909728 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:36:21.909754 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:36:21.911791 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:36:21.914318 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:21.914326 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:21.914343 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:21.914780 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:21.915095 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:21.915100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:21.915104 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:21.915307 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:21.915812 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:36:21.915819 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:36:21.916149 (+ 330us) client_negotiation.cc:769] Sending connection context
0618 22:36:21.916398 (+ 249us) client_negotiation.cc:240] Negotiation successful
0618 22:36:21.916894 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":253,"threads_started":1}
I20260618 22:36:21.919394 2541 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46483
I20260618 22:36:21.919696 2541 heartbeater.cc:464] Registering TS with master...
I20260618 22:36:21.920411 2541 heartbeater.cc:511] Master 127.18.150.190:46483 requested a full tablet report, sending...
I20260618 22:36:21.921744 2084 ts_manager.cc:194] Registered new tserver with Master: f54f490a53f94619a386da91da12c63c (127.18.150.131:38409)
I20260618 22:36:21.922610 2084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:51127
I20260618 22:36:21.931594 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:36:21.952739 2276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:21.935763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38140 (local address 127.18.150.190:46483)
0618 22:36:21.935904 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:21.935908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:21.936364 (+ 456us) server_negotiation.cc:407] Connection header received
0618 22:36:21.936511 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:21.936514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:21.936594 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:21.936666 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:21.937988 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.938829 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.940255 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.940647 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.940732 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:21.948153 (+ 7421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:21.948159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:21.948162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:21.948193 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:21.950590 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:21.951236 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:21.951239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:21.951242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:21.951360 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:21.951645 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:21.951648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:21.951650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:21.951936 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:21.952123 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:21.952374 (+ 251us) server_negotiation.cc:299] Negotiation successful
0618 22:36:21.952535 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:36:21.956549 2084 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:36:21.963138 2084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:36:21.976207 2274 heartbeater.cc:503] Master 127.18.150.190:46483 was elected leader, sending a full tablet report...
I20260618 22:36:21.996599 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:21.974458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:38409 (local address 127.0.0.1:48116)
0618 22:36:21.975173 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:21.975224 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:21.975402 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:21.989290 (+ 13888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:21.989297 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:21.989324 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:21.989740 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:21.989754 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.991636 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.991642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:21.993990 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:21.994008 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.995617 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.995624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:21.995811 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:21.995863 (+ 52us) client_negotiation.cc:769] Sending connection context
0618 22:36:21.995977 (+ 114us) client_negotiation.cc:240] Negotiation successful
0618 22:36:21.996166 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":276,"threads_started":1}
I20260618 22:36:21.997049 2557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:21.974876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:40131 (local address 127.0.0.1:53118)
0618 22:36:21.988164 (+ 13288us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:21.988191 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:21.988355 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:21.990589 (+ 2234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:21.990594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:21.990620 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:21.991147 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:21.991160 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.992678 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.992683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:21.994457 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:21.994475 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.996265 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.996272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:21.996419 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:21.996451 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:36:21.996530 (+ 79us) client_negotiation.cc:240] Negotiation successful
0618 22:36:21.996678 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13103,"mutex_wait_us":463,"thread_start_us":13593,"threads_started":1}
I20260618 22:36:21.997437 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:21.974599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48116 (local address 127.18.150.131:38409)
0618 22:36:21.988582 (+ 13983us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:21.988592 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:21.988631 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:36:21.988743 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:21.988751 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:21.989041 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:21.989259 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:21.989930 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.991506 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.994152 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.995439 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.995614 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:21.996884 (+ 1270us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:21.996952 (+ 68us) server_negotiation.cc:299] Negotiation successful
0618 22:36:21.997119 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13791,"thread_start_us":13644,"threads_started":1}
I20260618 22:36:21.997771 2555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:21.975194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53118 (local address 127.18.150.130:40131)
0618 22:36:21.989753 (+ 14559us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:21.989762 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:21.989789 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:36:21.989882 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:21.989889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:21.990172 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:21.990358 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:21.991329 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.992529 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.994625 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.995817 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.995963 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:21.996127 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:21.997345 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0618 22:36:21.997466 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14402,"thread_start_us":14294,"threads_started":1}
I20260618 22:36:21.997661 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:21.975503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:38009 (local address 127.0.0.1:59754)
0618 22:36:21.989147 (+ 13644us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:21.989162 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:21.989282 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:21.990552 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:21.990557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:21.990578 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:21.991164 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:21.991177 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.992926 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.992932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:21.994792 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:21.994805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.996906 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.996912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:21.997085 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:21.997115 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:36:21.997198 (+ 83us) client_negotiation.cc:240] Negotiation successful
0618 22:36:21.997325 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13551,"mutex_wait_us":52,"thread_start_us":12316,"threads_started":1}
I20260618 22:36:21.998235 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:21.975912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.18.150.129:38009)
0618 22:36:21.988891 (+ 12979us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:21.988900 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:21.989743 (+ 843us) server_negotiation.cc:407] Connection header received
0618 22:36:21.989848 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:21.989856 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:21.990127 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:21.990315 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:21.991368 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.992782 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.994962 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:21.996386 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:21.996588 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:21.996753 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:21.997772 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0618 22:36:21.997922 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12821,"thread_start_us":12672,"threads_started":1}
I20260618 22:36:22.001209 2476 tablet_service.cc:1467] Processing CreateTablet for tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04 (DEFAULT_TABLE table=test-table [id=fdd8a7841d154b248ba3a4d99167ede8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:36:22.001462 2209 tablet_service.cc:1467] Processing CreateTablet for tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04 (DEFAULT_TABLE table=test-table [id=fdd8a7841d154b248ba3a4d99167ede8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:36:22.001461 2343 tablet_service.cc:1467] Processing CreateTablet for tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04 (DEFAULT_TABLE table=test-table [id=fdd8a7841d154b248ba3a4d99167ede8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:36:22.002455 2476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:36:22.002796 2343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:36:22.002795 2209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:36:22.013240 2559 tablet_bootstrap.cc:492] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c: Bootstrap starting.
I20260618 22:36:22.016577 2559 tablet_bootstrap.cc:654] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c: Neither blocks nor log segments found. Creating new log.
I20260618 22:36:22.018273 2559 log.cc:824] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:22.020825 2560 tablet_bootstrap.cc:492] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3: Bootstrap starting.
I20260618 22:36:22.021106 2561 tablet_bootstrap.cc:492] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62: Bootstrap starting.
I20260618 22:36:22.021353 2559 tablet_bootstrap.cc:492] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c: No bootstrap required, opened a new log
I20260618 22:36:22.021615 2559 ts_tablet_manager.cc:1397] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260618 22:36:22.023763 2561 tablet_bootstrap.cc:654] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62: Neither blocks nor log segments found. Creating new log.
I20260618 22:36:22.024969 2561 log.cc:824] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:22.027084 2560 tablet_bootstrap.cc:654] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3: Neither blocks nor log segments found. Creating new log.
I20260618 22:36:22.027974 2561 tablet_bootstrap.cc:492] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62: No bootstrap required, opened a new log
I20260618 22:36:22.028072 2560 log.cc:824] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:22.028240 2561 ts_tablet_manager.cc:1397] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260618 22:36:22.030427 2560 tablet_bootstrap.cc:492] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3: No bootstrap required, opened a new log
I20260618 22:36:22.030148 2559 raft_consensus.cc:348] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:22.030551 2559 raft_consensus.cc:374] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:36:22.030665 2560 ts_tablet_manager.cc:1397] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260618 22:36:22.030663 2559 raft_consensus.cc:729] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f54f490a53f94619a386da91da12c63c, State: Initialized, Role: FOLLOWER
I20260618 22:36:22.031239 2559 consensus_queue.cc:260] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:22.032255 2541 heartbeater.cc:503] Master 127.18.150.190:46483 was elected leader, sending a full tablet report...
I20260618 22:36:22.032895 2559 ts_tablet_manager.cc:1428] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260618 22:36:22.038096 2561 raft_consensus.cc:348] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:22.038528 2561 raft_consensus.cc:374] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:36:22.038622 2561 raft_consensus.cc:729] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6d9ff501c942cca6dae0246ce97c62, State: Initialized, Role: FOLLOWER
I20260618 22:36:22.039198 2561 consensus_queue.cc:260] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:22.040166 2408 heartbeater.cc:503] Master 127.18.150.190:46483 was elected leader, sending a full tablet report...
I20260618 22:36:22.040867 2561 ts_tablet_manager.cc:1428] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260618 22:36:22.041283 2560 raft_consensus.cc:348] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:22.041604 2560 raft_consensus.cc:374] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:36:22.041734 2560 raft_consensus.cc:729] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea081ab812f14c4dbbc7529e044f30d3, State: Initialized, Role: FOLLOWER
I20260618 22:36:22.042375 2560 consensus_queue.cc:260] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:22.043860 2560 ts_tablet_manager.cc:1428] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260618 22:36:22.080684 2567 raft_consensus.cc:482] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:36:22.081005 2567 raft_consensus.cc:504] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:22.082784 2567 leader_election.cc:288] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f54f490a53f94619a386da91da12c63c (127.18.150.131:38409), bd6d9ff501c942cca6dae0246ce97c62 (127.18.150.130:40131)
I20260618 22:36:22.089358 2555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:22.083210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:46847 (local address 127.18.150.130:40131)
0618 22:36:22.083441 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:22.083448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:22.083848 (+ 400us) server_negotiation.cc:407] Connection header received
0618 22:36:22.084058 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:22.084062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:22.084125 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:22.084213 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:22.084908 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.085790 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.087706 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.088448 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.088590 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:22.088665 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:22.088992 (+ 327us) server_negotiation.cc:299] Negotiation successful
0618 22:36:22.089121 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:36:22.089390 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:22.083346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:38509 (local address 127.18.150.131:38409)
0618 22:36:22.083553 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:22.083559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:22.084077 (+ 518us) server_negotiation.cc:407] Connection header received
0618 22:36:22.084250 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:22.084253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:22.084313 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:22.084381 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:22.085085 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.086007 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.087696 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.088460 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.088603 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:22.088675 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:22.089035 (+ 360us) server_negotiation.cc:299] Negotiation successful
0618 22:36:22.089155 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:36:22.089715 2569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:22.083136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:38409 (local address 127.18.150.129:38509)
0618 22:36:22.083997 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:22.084012 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:22.084123 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:22.084417 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:22.084421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:22.084482 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:22.084909 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:22.084921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.086140 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.086144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:22.087546 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:22.087558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.088712 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.088719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:22.088875 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:22.088897 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:36:22.088984 (+ 87us) client_negotiation.cc:240] Negotiation successful
0618 22:36:22.089102 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":18,"spinlock_wait_cycles":36224,"thread_start_us":565,"threads_started":1}
I20260618 22:36:22.089754 2568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:22.083058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:40131 (local address 127.18.150.129:46847)
0618 22:36:22.083721 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:22.083745 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:22.083922 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:22.084257 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:22.084261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:22.084288 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:22.084689 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:22.084705 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.085924 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.085930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:22.087555 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:22.087567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.088673 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.088680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:22.088817 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:22.088873 (+ 56us) client_negotiation.cc:769] Sending connection context
0618 22:36:22.088945 (+ 72us) client_negotiation.cc:240] Negotiation successful
0618 22:36:22.089068 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":70,"thread_start_us":386,"threads_started":1}
I20260618 22:36:22.090642 2363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e1ad2aa0d634c7d98ab1bb8b9125a04" candidate_uuid: "ea081ab812f14c4dbbc7529e044f30d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6d9ff501c942cca6dae0246ce97c62" is_pre_election: true
I20260618 22:36:22.090646 2496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e1ad2aa0d634c7d98ab1bb8b9125a04" candidate_uuid: "ea081ab812f14c4dbbc7529e044f30d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f54f490a53f94619a386da91da12c63c" is_pre_election: true
I20260618 22:36:22.091089 2363 raft_consensus.cc:2454] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea081ab812f14c4dbbc7529e044f30d3 in term 0.
I20260618 22:36:22.091087 2496 raft_consensus.cc:2454] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea081ab812f14c4dbbc7529e044f30d3 in term 0.
I20260618 22:36:22.091676 2162 leader_election.cc:302] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea081ab812f14c4dbbc7529e044f30d3, f54f490a53f94619a386da91da12c63c; no voters:
I20260618 22:36:22.092048 2567 raft_consensus.cc:2781] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:36:22.092167 2567 raft_consensus.cc:482] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:36:22.092259 2567 raft_consensus.cc:3037] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:36:22.094420 2567 raft_consensus.cc:504] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:22.095012 2567 leader_election.cc:288] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [CANDIDATE]: Term 1 election: Requested vote from peers f54f490a53f94619a386da91da12c63c (127.18.150.131:38409), bd6d9ff501c942cca6dae0246ce97c62 (127.18.150.130:40131)
I20260618 22:36:22.095391 2496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e1ad2aa0d634c7d98ab1bb8b9125a04" candidate_uuid: "ea081ab812f14c4dbbc7529e044f30d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f54f490a53f94619a386da91da12c63c"
I20260618 22:36:22.095453 2363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e1ad2aa0d634c7d98ab1bb8b9125a04" candidate_uuid: "ea081ab812f14c4dbbc7529e044f30d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6d9ff501c942cca6dae0246ce97c62"
I20260618 22:36:22.095585 2496 raft_consensus.cc:3037] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:36:22.095656 2363 raft_consensus.cc:3037] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:36:22.097795 2496 raft_consensus.cc:2454] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea081ab812f14c4dbbc7529e044f30d3 in term 1.
I20260618 22:36:22.097793 2363 raft_consensus.cc:2454] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea081ab812f14c4dbbc7529e044f30d3 in term 1.
I20260618 22:36:22.098160 2164 leader_election.cc:302] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6d9ff501c942cca6dae0246ce97c62, ea081ab812f14c4dbbc7529e044f30d3; no voters:
I20260618 22:36:22.098454 2567 raft_consensus.cc:2781] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:36:22.098847 2567 raft_consensus.cc:686] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 1 LEADER]: Becoming Leader. State: Replica: ea081ab812f14c4dbbc7529e044f30d3, State: Running, Role: LEADER
I20260618 22:36:22.099332 2567 consensus_queue.cc:237] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:22.103579 2085 catalog_manager.cc:5495] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea081ab812f14c4dbbc7529e044f30d3 (127.18.150.129). New cstate: current_term: 1 leader_uuid: "ea081ab812f14c4dbbc7529e044f30d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } health_report { overall_health: UNKNOWN } } }
I20260618 22:36:22.120755 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:22.115614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.18.150.129:38009)
0618 22:36:22.115766 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:22.115770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:22.115996 (+ 226us) server_negotiation.cc:407] Connection header received
0618 22:36:22.116178 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:22.116182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:22.116303 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:22.116419 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:36:22.116735 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.117440 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.118822 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.119222 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.119309 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:22.119750 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:36:22.120290 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:36:22.120392 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:22.120485 (+ 93us) server_negotiation.cc:299] Negotiation successful
0618 22:36:22.120550 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:36:22.128536 2363 raft_consensus.cc:1261] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62 [term 1 FOLLOWER]: Refusing update from remote peer ea081ab812f14c4dbbc7529e044f30d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:36:22.128548 2496 raft_consensus.cc:1261] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c [term 1 FOLLOWER]: Refusing update from remote peer ea081ab812f14c4dbbc7529e044f30d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:36:22.129386 2567 consensus_queue.cc:1035] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:36:22.129648 2570 consensus_queue.cc:1035] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:36:22.156796 2084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:36:22.157068 2084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:36:22.160501 2084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Sending DeleteTablet for 3 replicas of tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04
I20260618 22:36:22.161600 2476 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:36:22 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:48116
I20260618 22:36:22.161705 2343 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:36:22 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:53118
I20260618 22:36:22.161928 2209 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:36:22 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:59754
I20260618 22:36:22.162734 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 2146
I20260618 22:36:22.172567 2585 tablet_replica.cc:330] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3: stopping tablet replica
I20260618 22:36:22.173060 2585 raft_consensus.cc:2229] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:36:22.173729 2585 raft_consensus.cc:2258] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:36:22.178086 2585 ts_tablet_manager.cc:1905] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:36:22.179605 2583 tablet_replica.cc:330] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c: stopping tablet replica
I20260618 22:36:22.180053 2583 raft_consensus.cc:2229] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:36:22.180683 2583 raft_consensus.cc:2258] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:36:22.182823 2583 ts_tablet_manager.cc:1905] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:36:22.184808 2584 tablet_replica.cc:330] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62: stopping tablet replica
I20260618 22:36:22.185317 2584 raft_consensus.cc:2229] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:36:22.185806 2584 raft_consensus.cc:2258] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:36:22.187824 2584 ts_tablet_manager.cc:1905] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 18 22:36:22 dist-test-slave-h7jz krb5kdc[2040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:36:22.189780 2583 ts_tablet_manager.cc:1918] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:36:22.189920 2583 log.cc:1196] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/6e1ad2aa0d634c7d98ab1bb8b9125a04
I20260618 22:36:22.190478 2583 ts_tablet_manager.cc:1939] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P f54f490a53f94619a386da91da12c63c: Deleting consensus metadata
I20260618 22:36:22.192315 2069 catalog_manager.cc:4841] TS f54f490a53f94619a386da91da12c63c (127.18.150.131:38409): tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04 (table test-table [id=fdd8a7841d154b248ba3a4d99167ede8]) successfully deleted
I20260618 22:36:22.194424 2584 ts_tablet_manager.cc:1918] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:36:22.194589 2584 log.cc:1196] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/6e1ad2aa0d634c7d98ab1bb8b9125a04
I20260618 22:36:22.195117 2584 ts_tablet_manager.cc:1939] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P bd6d9ff501c942cca6dae0246ce97c62: Deleting consensus metadata
I20260618 22:36:22.196164 2585 ts_tablet_manager.cc:1918] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:36:22.196377 2585 log.cc:1196] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/6e1ad2aa0d634c7d98ab1bb8b9125a04
I20260618 22:36:22.196909 2585 ts_tablet_manager.cc:1939] T 6e1ad2aa0d634c7d98ab1bb8b9125a04 P ea081ab812f14c4dbbc7529e044f30d3: Deleting consensus metadata
I20260618 22:36:22.196988 2071 catalog_manager.cc:4841] TS bd6d9ff501c942cca6dae0246ce97c62 (127.18.150.130:40131): tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04 (table test-table [id=fdd8a7841d154b248ba3a4d99167ede8]) successfully deleted
I20260618 22:36:22.198463 2069 catalog_manager.cc:4841] TS ea081ab812f14c4dbbc7529e044f30d3 (127.18.150.129:38009): tablet 6e1ad2aa0d634c7d98ab1bb8b9125a04 (table test-table [id=fdd8a7841d154b248ba3a4d99167ede8]) successfully deleted
I20260618 22:36:22.199606 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:22.163513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59772 (local address 127.18.150.129:38009)
0618 22:36:22.163689 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:22.163695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:22.178806 (+ 15111us) server_negotiation.cc:407] Connection header received
0618 22:36:22.179045 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:22.179051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:22.179135 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:22.179246 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:22.181321 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.182648 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.184664 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.185548 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.185653 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:22.191203 (+ 5550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:22.191217 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:22.191234 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:22.191285 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:22.196011 (+ 4726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:22.197024 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:22.197033 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:22.197037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:22.197216 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:22.197677 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:22.197682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:22.197685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:22.198203 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:22.198493 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:22.199156 (+ 663us) server_negotiation.cc:299] Negotiation successful
0618 22:36:22.199336 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:36:22.200274 2269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:22.296909 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 2146
I20260618 22:36:22.312199 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 2280
Jun 18 22:36:22 dist-test-slave-h7jz krb5kdc[2040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:36:22.327643 2555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:22.313030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53120 (local address 127.18.150.130:40131)
0618 22:36:22.313217 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:22.313223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:22.313356 (+ 133us) server_negotiation.cc:407] Connection header received
0618 22:36:22.313572 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:22.313579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:22.313680 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:22.313757 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:22.315643 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.316626 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.318305 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.318838 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.318935 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:22.322949 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:22.322963 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:22.322978 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:22.323013 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:22.325388 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:22.325997 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:22.326003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:22.326006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:22.326114 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:22.326406 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:22.326409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:22.326411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:22.326764 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:22.326972 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:22.327179 (+ 207us) server_negotiation.cc:299] Negotiation successful
0618 22:36:22.327334 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:36:22.328279 2403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:22.427932 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 2280
I20260618 22:36:22.441408 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 2413
Jun 18 22:36:22 dist-test-slave-h7jz krb5kdc[2040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:36:22.456328 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:22.442263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48124 (local address 127.18.150.131:38409)
0618 22:36:22.442439 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:22.442444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:22.442462 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:36:22.442577 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:22.442581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:22.442645 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:22.442717 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:22.443989 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.444741 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.446163 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:22.446547 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:22.446644 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:22.450945 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:22.450961 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:22.450977 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:22.451013 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:22.453653 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:22.454523 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:22.454530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:22.454532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:22.454675 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:22.455052 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:22.455055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:22.455057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:22.455388 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:22.455612 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:22.455879 (+ 267us) server_negotiation.cc:299] Negotiation successful
0618 22:36:22.456042 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:36:22.456972 2536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:22.544516 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 2413
I20260618 22:36:22.557389 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 2053
I20260618 22:36:22.558455 2115 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:22.666801 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 2053
I20260618 22:36:42.684911 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:46483
--webserver_interface=127.18.150.190
--webserver_port=46055
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:46483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:36:43.001693 2591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:43.002013 2591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:43.002146 2591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:43.009205 2591 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260618 22:36:43.009306 2591 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260618 22:36:43.009370 2591 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260618 22:36:43.009439 2591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:36:43.009516 2591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:43.009608 2591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:43.009667 2591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:36:43.009742 2591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:36:43.019400 2591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:46483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:46483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=46055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:43.020639 2591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:43.022001 2591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:43.042887 2598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:43.042946 2597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:43.042922 2600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:43.043537 2591 server_base.cc:1034] running on GCE node
I20260618 22:36:43.044394 2591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:43.045909 2591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:43.047161 2591 hybrid_clock.cc:648] HybridClock initialized: now 1781822203047081 us; error 35 us; skew 500 ppm
I20260618 22:36:43.047525 2591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:43 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:43.053741 2591 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:36:43.055250 2591 webserver.cc:460] Webserver started at http://127.18.150.190:46055/ using document root <none> and password file <none>
I20260618 22:36:43.055968 2591 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:43.056077 2591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:43.061683 2591 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:36:43.064543 2607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:43.065930 2591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:36:43.066115 2591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "c70be62673614a6ea5293395381c1f55"
format_stamp: "Formatted at 2026-06-18 22:36:20 on dist-test-slave-h7jz"
server_key: "7a41200c303f1de255d1aff511d659ef"
server_key_iv: "86d8dadc71b30d35fa9137f892ac0af4"
server_key_version: "encryptionkey@0"
I20260618 22:36:43.067142 2591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:43.081493 2591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:43.085472 2591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:43.085736 2591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:43.102962 2591 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:46483
I20260618 22:36:43.102980 2658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:46483 every 8 connection(s)
I20260618 22:36:43.104460 2591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:36:43.105780 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 2591
I20260618 22:36:43.106781 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:38009
--local_ip_for_outbound_sockets=127.18.150.129
--tserver_master_addrs=127.18.150.190:46483
--webserver_port=36369
--webserver_interface=127.18.150.129
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:36:43.113478 2659 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:36:43.117273 2659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Bootstrap starting.
I20260618 22:36:43.136803 2659 log.cc:824] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:43.148659 2659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:36:43.149196 2659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Bootstrap complete.
I20260618 22:36:43.158277 2659 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:36:43.159108 2659 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70be62673614a6ea5293395381c1f55, State: Initialized, Role: FOLLOWER
I20260618 22:36:43.159744 2659 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:36:43.159953 2659 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:36:43.160069 2659 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:36:43.160224 2659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:36:43.163951 2659 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:36:43.164460 2659 leader_election.cc:302] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70be62673614a6ea5293395381c1f55; no voters:
I20260618 22:36:43.165061 2659 leader_election.cc:288] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:36:43.165244 2663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:36:43.166308 2663 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 2 LEADER]: Becoming Leader. State: Replica: c70be62673614a6ea5293395381c1f55, State: Running, Role: LEADER
I20260618 22:36:43.166882 2663 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:36:43.167240 2659 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:36:43.169232 2665 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70be62673614a6ea5293395381c1f55. Latest consensus state: current_term: 2 leader_uuid: "c70be62673614a6ea5293395381c1f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } } }
I20260618 22:36:43.169585 2665 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: This master's current role is: LEADER
I20260618 22:36:43.170284 2664 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c70be62673614a6ea5293395381c1f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } } }
I20260618 22:36:43.170567 2664 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: This master's current role is: LEADER
I20260618 22:36:43.180852 2670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:36:43.189330 2670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:36:43.190783 2670 catalog_manager.cc:1259] Loaded cluster ID: 18df41e3270940f1a382ae3e02efea5d
I20260618 22:36:43.190937 2670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:36:43.194825 2670 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:36:43.196301 2670 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Loaded TSK: 0
I20260618 22:36:43.238777 2670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Generated new TSK 1
I20260618 22:36:43.239171 2670 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:36:43.442413 2661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:43.442677 2661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:43.442788 2661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:43.450040 2661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:43.450153 2661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:43.450290 2661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:36:43.460307 2661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:38009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=36369
--tserver_master_addrs=127.18.150.190:46483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:43.461623 2661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:43.463007 2661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:43.476366 2687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:43.478451 2689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:43.478463 2686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:43.479270 2661 server_base.cc:1034] running on GCE node
I20260618 22:36:43.479848 2661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:43.480813 2661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:43.482048 2661 hybrid_clock.cc:648] HybridClock initialized: now 1781822203482004 us; error 36 us; skew 500 ppm
I20260618 22:36:43.482395 2661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:43 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:43.485350 2661 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:36:43.486866 2661 webserver.cc:460] Webserver started at http://127.18.150.129:36369/ using document root <none> and password file <none>
I20260618 22:36:43.487529 2661 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:43.487646 2661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:43.492960 2661 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:36:43.495743 2696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:43.497366 2661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:36:43.497536 2661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "ea081ab812f14c4dbbc7529e044f30d3"
format_stamp: "Formatted at 2026-06-18 22:36:20 on dist-test-slave-h7jz"
server_key: "db13b6a1ee42a98cf25cca7493c49d93"
server_key_iv: "eaf27c76f04b3c806ff5d131d44e305e"
server_key_version: "encryptionkey@0"
I20260618 22:36:43.498600 2661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:43.528757 2661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:43.533278 2661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:43.533568 2661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:43.534605 2661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:36:43.535951 2661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:36:43.536043 2661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:43.536144 2661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:36:43.536217 2661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:43.565466 2661 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:38009
I20260618 22:36:43.565519 2808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:38009 every 8 connection(s)
I20260618 22:36:43.566901 2661 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:36:43.571134 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 2661
I20260618 22:36:43.571913 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:40131
--local_ip_for_outbound_sockets=127.18.150.130
--tserver_master_addrs=127.18.150.190:46483
--webserver_port=36421
--webserver_interface=127.18.150.130
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:36:43 dist-test-slave-h7jz krb5kdc[2040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:36:43.591928 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:43.570115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:49047)
0618 22:36:43.570931 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:43.570997 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:43.573304 (+ 2307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:43.574751 (+ 1447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:43.574761 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:43.575454 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:36:43.576194 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:43.576218 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:43.577807 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:43.577812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:43.579539 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:43.579553 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:43.580951 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:43.580956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:43.581213 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:43.581877 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:36:43.581899 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:36:43.584171 (+ 2272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:36:43.588026 (+ 3855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:43.588033 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:43.588044 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:43.588442 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:43.588962 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:43.588966 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:43.588969 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:43.589196 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:43.590082 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:36:43.590088 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:36:43.590373 (+ 285us) client_negotiation.cc:769] Sending connection context
0618 22:36:43.590595 (+ 222us) client_negotiation.cc:240] Negotiation successful
0618 22:36:43.590973 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":273,"threads_started":1}
I20260618 22:36:43.592990 2811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:43.569776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:49047 (local address 127.18.150.190:46483)
0618 22:36:43.570345 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:43.570357 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:43.571232 (+ 875us) server_negotiation.cc:407] Connection header received
0618 22:36:43.573843 (+ 2611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:43.573856 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:43.574262 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:43.574571 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:43.576405 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:43.577650 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:43.579706 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:43.580754 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:43.580927 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:43.584452 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:43.584461 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:43.584480 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:43.584524 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:43.587899 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:43.588596 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:43.588605 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:43.588612 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:43.588856 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:43.589304 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:43.589308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:43.589311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:43.589935 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:43.590153 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:43.590878 (+ 725us) server_negotiation.cc:299] Negotiation successful
0618 22:36:43.591320 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":256,"threads_started":1}
I20260618 22:36:43.595920 2809 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46483
I20260618 22:36:43.596191 2809 heartbeater.cc:464] Registering TS with master...
I20260618 22:36:43.596855 2809 heartbeater.cc:511] Master 127.18.150.190:46483 requested a full tablet report, sending...
I20260618 22:36:43.598862 2624 ts_manager.cc:194] Registered new tserver with Master: ea081ab812f14c4dbbc7529e044f30d3 (127.18.150.129:38009)
I20260618 22:36:43.600288 2624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:49047
W20260618 22:36:43.893641 2813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:43.893906 2813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:43.894062 2813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:43.901271 2813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:43.901403 2813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:43.901563 2813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:36:43.911780 2813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:40131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=36421
--tserver_master_addrs=127.18.150.190:46483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:43.913079 2813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:43.914436 2813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:43.927341 2818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:43.927294 2819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:43.927271 2821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:43.928402 2813 server_base.cc:1034] running on GCE node
I20260618 22:36:43.929006 2813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:43.929908 2813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:43.931183 2813 hybrid_clock.cc:648] HybridClock initialized: now 1781822203931101 us; error 41 us; skew 500 ppm
I20260618 22:36:43.931512 2813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:43 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:43.934659 2813 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:36:43.936029 2813 webserver.cc:460] Webserver started at http://127.18.150.130:36421/ using document root <none> and password file <none>
I20260618 22:36:43.936710 2813 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:43.936829 2813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:43.941982 2813 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:36:43.944528 2828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:43.945859 2813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260618 22:36:43.946024 2813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "bd6d9ff501c942cca6dae0246ce97c62"
format_stamp: "Formatted at 2026-06-18 22:36:21 on dist-test-slave-h7jz"
server_key: "8c302273fc18059616487156161de372"
server_key_iv: "a96de743e63b45906cea1aec2fe6dfa2"
server_key_version: "encryptionkey@0"
I20260618 22:36:43.946944 2813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:43.986938 2813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:43.990823 2813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:43.991070 2813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:43.992009 2813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:36:43.993315 2813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:36:43.993410 2813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:43.993513 2813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:36:43.993587 2813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:44.023381 2813 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:40131
I20260618 22:36:44.023444 2940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:40131 every 8 connection(s)
I20260618 22:36:44.024752 2813 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:36:44.028357 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 2813
I20260618 22:36:44.029110 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:38409
--local_ip_for_outbound_sockets=127.18.150.131
--tserver_master_addrs=127.18.150.190:46483
--webserver_port=35203
--webserver_interface=127.18.150.131
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:36:44 dist-test-slave-h7jz krb5kdc[2040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:36:44.048976 2811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.027133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:42525 (local address 127.18.150.190:46483)
0618 22:36:44.027304 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:44.027309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:44.028646 (+ 1337us) server_negotiation.cc:407] Connection header received
0618 22:36:44.031055 (+ 2409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:44.031060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:44.031133 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:44.031224 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:44.033104 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.034059 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.039074 (+ 5015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.039501 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.039594 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.043444 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:44.043449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:44.043452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:44.043486 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:44.045859 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:44.046703 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:44.046707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:44.046709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:44.046823 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:44.047273 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:44.047277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:44.047280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:44.047604 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:44.047721 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:44.048564 (+ 843us) server_negotiation.cc:299] Negotiation successful
0618 22:36:44.048747 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:36:44.050726 2943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.027645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:42525)
0618 22:36:44.028461 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:44.028510 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:44.030771 (+ 2261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:44.031466 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:44.031477 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:44.032161 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:36:44.032913 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.032940 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.037498 (+ 4558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.037502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.038920 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.038934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.039706 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.039712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.039976 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.040655 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:36:44.040681 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:36:44.043224 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:36:44.046017 (+ 2793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:44.046026 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:44.046044 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:44.046604 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:44.046920 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:44.046925 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:44.046928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:44.047164 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:44.047732 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:36:44.047740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:36:44.048195 (+ 455us) client_negotiation.cc:769] Sending connection context
0618 22:36:44.048516 (+ 321us) client_negotiation.cc:240] Negotiation successful
0618 22:36:44.049047 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":282,"threads_started":1}
I20260618 22:36:44.052166 2941 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46483
I20260618 22:36:44.052511 2941 heartbeater.cc:464] Registering TS with master...
I20260618 22:36:44.053438 2941 heartbeater.cc:511] Master 127.18.150.190:46483 requested a full tablet report, sending...
I20260618 22:36:44.055080 2624 ts_manager.cc:194] Registered new tserver with Master: bd6d9ff501c942cca6dae0246ce97c62 (127.18.150.130:40131)
I20260618 22:36:44.056185 2624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:42525
W20260618 22:36:44.351218 2944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:36:44.351483 2944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:36:44.351579 2944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:36:44.358654 2944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:36:44.358762 2944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:36:44.358875 2944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:36:44.368479 2944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:38409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=35203
--tserver_master_addrs=127.18.150.190:46483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:36:44.369793 2944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:36:44.371130 2944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:36:44.384169 2949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:44.384148 2950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:36:44.384120 2952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:36:44.385013 2944 server_base.cc:1034] running on GCE node
I20260618 22:36:44.385599 2944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:36:44.386600 2944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:36:44.387791 2944 hybrid_clock.cc:648] HybridClock initialized: now 1781822204387737 us; error 55 us; skew 500 ppm
I20260618 22:36:44.388087 2944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:36:44 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822204, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:36:44.391214 2944 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:36:44.392645 2944 webserver.cc:460] Webserver started at http://127.18.150.131:35203/ using document root <none> and password file <none>
I20260618 22:36:44.393309 2944 fs_manager.cc:359] Metadata directory not provided
I20260618 22:36:44.393436 2944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:36:44.398989 2944 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:36:44.401548 2959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:44.402910 2944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:36:44.403055 2944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "f54f490a53f94619a386da91da12c63c"
format_stamp: "Formatted at 2026-06-18 22:36:21 on dist-test-slave-h7jz"
server_key: "eac2db335a955438425ebfe7bfd18879"
server_key_iv: "cc8c3b2f19f111b6ffa76cafce5f697c"
server_key_version: "encryptionkey@0"
I20260618 22:36:44.403976 2944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:36:44.417016 2944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:36:44.420831 2944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:36:44.421082 2944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:36:44.422029 2944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:36:44.423305 2944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:36:44.423403 2944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:44.423507 2944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:36:44.423575 2944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:36:44.452780 2944 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:38409
I20260618 22:36:44.452844 3071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:38409 every 8 connection(s)
I20260618 22:36:44.454202 2944 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:36:44.463966 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 2944
Jun 18 22:36:44 dist-test-slave-h7jz krb5kdc[2040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822204, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:36:44.474009 2811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.456618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:42823 (local address 127.18.150.190:46483)
0618 22:36:44.456897 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:44.456902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:44.457904 (+ 1002us) server_negotiation.cc:407] Connection header received
0618 22:36:44.459747 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:44.459752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:44.459833 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:44.459918 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:44.461370 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.462126 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.463637 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.464031 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.464144 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.468183 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:44.468187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:44.468190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:44.468227 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:44.471014 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:44.471748 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:44.471754 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:44.471757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:44.471888 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:44.472309 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:44.472313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:44.472315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:44.472640 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:44.472772 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:44.473416 (+ 644us) server_negotiation.cc:299] Negotiation successful
0618 22:36:44.473629 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260618 22:36:44.475174 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.457062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:42823)
0618 22:36:44.457745 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:44.457782 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:44.459486 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:44.460161 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:44.460169 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:44.460667 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:36:44.461211 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.461228 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.462281 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.462285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.463480 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.463492 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.464319 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.464327 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.464593 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.465271 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:36:44.465303 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:36:44.467961 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:36:44.471151 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:44.471159 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:44.471172 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:44.471624 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:44.471992 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:36:44.471997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:36:44.472001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:36:44.472203 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:36:44.472759 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:36:44.472767 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:36:44.473084 (+ 317us) client_negotiation.cc:769] Sending connection context
0618 22:36:44.473318 (+ 234us) client_negotiation.cc:240] Negotiation successful
0618 22:36:44.473845 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":250,"threads_started":1}
I20260618 22:36:44.476596 3072 heartbeater.cc:347] Connected to a master server at 127.18.150.190:46483
I20260618 22:36:44.476919 3072 heartbeater.cc:464] Registering TS with master...
I20260618 22:36:44.477802 3072 heartbeater.cc:511] Master 127.18.150.190:46483 requested a full tablet report, sending...
I20260618 22:36:44.479952 2624 ts_manager.cc:194] Registered new tserver with Master: f54f490a53f94619a386da91da12c63c (127.18.150.131:38409)
I20260618 22:36:44.480966 2624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:42823
I20260618 22:36:44.481218 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.464884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56010 (local address 127.18.150.190:46483)
0618 22:36:44.465487 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:44.465493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:44.466450 (+ 957us) server_negotiation.cc:407] Connection header received
0618 22:36:44.466619 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:44.466624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:44.466695 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:44.466801 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:44.468645 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.469774 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.471244 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.471790 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.471898 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.474436 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:44.474442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:44.474447 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:44.474480 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:44.477409 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:44.478279 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:44.478284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:44.478287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:44.478414 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:44.478873 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:44.478877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:44.478879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:44.479194 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:44.479445 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:44.480575 (+ 1130us) server_negotiation.cc:299] Negotiation successful
0618 22:36:44.480803 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":350,"threads_started":1}
I20260618 22:36:44.483479 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:36:44.489305 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.484305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56012 (local address 127.18.150.190:46483)
0618 22:36:44.484443 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:44.484446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:44.484833 (+ 387us) server_negotiation.cc:407] Connection header received
0618 22:36:44.484986 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:44.484990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:44.485109 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:44.485218 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:36:44.485556 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.486278 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.487785 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.488176 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.488262 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.488675 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:36:44.488852 (+ 177us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0618 22:36:44.489111 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56012: token expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260618 22:36:44.489399 3075 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56012: token expired
W20260618 22:36:44.489903 2546 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.150.190:46483)
I20260618 22:36:44.500742 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.490322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56020 (local address 127.18.150.190:46483)
0618 22:36:44.490440 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:44.490444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:44.490487 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:36:44.490596 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:44.490599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:44.490657 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:44.490725 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:44.491848 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.492508 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.493675 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.494090 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.494170 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.496503 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:44.496507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:44.496510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:44.496537 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:44.498825 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:44.499337 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:44.499340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:44.499342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:44.499447 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:44.499707 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:44.499709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:44.499711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:44.499974 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:44.500142 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:44.500361 (+ 219us) server_negotiation.cc:299] Negotiation successful
0618 22:36:44.500506 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:36:44.503882 2624 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:36:44.511103 2624 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:36:44.535225 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.522713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48026 (local address 127.18.150.130:40131)
0618 22:36:44.527347 (+ 4634us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:44.527354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:44.527380 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:36:44.528065 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:44.528074 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:44.528466 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:44.529236 (+ 770us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:44.529505 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.530697 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.532820 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.534066 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.534249 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.534424 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:44.534701 (+ 277us) server_negotiation.cc:299] Negotiation successful
0618 22:36:44.534867 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4484,"thread_start_us":4346,"threads_started":1}
I20260618 22:36:44.536091 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.524217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52646 (local address 127.18.150.129:38009)
0618 22:36:44.527091 (+ 2874us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:44.527100 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:44.527139 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:36:44.527238 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:44.527245 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:44.527501 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:44.527686 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:44.528617 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.530136 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.532996 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.534208 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.534366 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.534524 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:44.535233 (+ 709us) server_negotiation.cc:299] Negotiation successful
0618 22:36:44.535817 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2706,"thread_start_us":2608,"threads_started":1}
I20260618 22:36:44.536128 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.522076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55114 (local address 127.18.150.131:38409)
0618 22:36:44.527865 (+ 5789us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:44.527870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:44.527894 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:36:44.527970 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:44.527976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:44.528217 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:44.528406 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:44.529012 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.530063 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.533589 (+ 3526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.534890 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.535061 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.535212 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:44.535720 (+ 508us) server_negotiation.cc:299] Negotiation successful
0618 22:36:44.535866 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5653,"thread_start_us":5513,"threads_started":1}
I20260618 22:36:44.536211 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.522295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:40131 (local address 127.0.0.1:48026)
0618 22:36:44.526219 (+ 3924us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:44.526239 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:44.526355 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:44.528671 (+ 2316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:44.528676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:44.528702 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:44.529065 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.529076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.530816 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.530822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.532680 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.532692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.534267 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.534273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.534461 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.534510 (+ 49us) client_negotiation.cc:769] Sending connection context
0618 22:36:44.534665 (+ 155us) client_negotiation.cc:240] Negotiation successful
0618 22:36:44.534853 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1588,"mutex_wait_us":3,"thread_start_us":880,"threads_started":1}
I20260618 22:36:44.536580 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.523859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:38009 (local address 127.0.0.1:52646)
0618 22:36:44.526785 (+ 2926us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:44.526802 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:44.526903 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:44.527960 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:44.527968 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:44.527996 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:44.528394 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.528407 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.530235 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.530240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.532808 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.532823 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.534676 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.534682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.534857 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.534885 (+ 28us) client_negotiation.cc:769] Sending connection context
0618 22:36:44.534965 (+ 80us) client_negotiation.cc:240] Negotiation successful
0618 22:36:44.535077 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2826,"mutex_wait_us":124,"thread_start_us":2664,"threads_started":1}
I20260618 22:36:44.536988 3084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.521865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:38409 (local address 127.0.0.1:55114)
0618 22:36:44.523410 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:44.523451 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:44.523599 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:44.528404 (+ 4805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:44.528408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:44.528432 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:44.528777 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.528787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.530201 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.530206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.533379 (+ 3173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.533392 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.535369 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.535376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.535539 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.535562 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:36:44.535639 (+ 77us) client_negotiation.cc:240] Negotiation successful
0618 22:36:44.535758 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1305,"mutex_wait_us":45,"thread_start_us":1180,"threads_started":1}
I20260618 22:36:44.539968 2744 tablet_service.cc:1467] Processing CreateTablet for tablet 3eab06a46cf640a9bb7272c6de8cba69 (DEFAULT_TABLE table=test-table [id=14e85ca4ec744489bba318b45818e4c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:36:44.539948 3007 tablet_service.cc:1467] Processing CreateTablet for tablet 3eab06a46cf640a9bb7272c6de8cba69 (DEFAULT_TABLE table=test-table [id=14e85ca4ec744489bba318b45818e4c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:36:44.539968 2876 tablet_service.cc:1467] Processing CreateTablet for tablet 3eab06a46cf640a9bb7272c6de8cba69 (DEFAULT_TABLE table=test-table [id=14e85ca4ec744489bba318b45818e4c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:36:44.541334 3007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eab06a46cf640a9bb7272c6de8cba69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:36:44.541360 2744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eab06a46cf640a9bb7272c6de8cba69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:36:44.541356 2876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eab06a46cf640a9bb7272c6de8cba69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:36:44.552918 3088 tablet_bootstrap.cc:492] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: Bootstrap starting.
I20260618 22:36:44.556119 3089 tablet_bootstrap.cc:492] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: Bootstrap starting.
I20260618 22:36:44.557359 3088 tablet_bootstrap.cc:654] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: Neither blocks nor log segments found. Creating new log.
I20260618 22:36:44.557542 3090 tablet_bootstrap.cc:492] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: Bootstrap starting.
I20260618 22:36:44.558777 3088 log.cc:824] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:44.559526 3089 tablet_bootstrap.cc:654] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: Neither blocks nor log segments found. Creating new log.
I20260618 22:36:44.560320 3090 tablet_bootstrap.cc:654] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: Neither blocks nor log segments found. Creating new log.
I20260618 22:36:44.560556 3089 log.cc:824] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:44.561301 3090 log.cc:824] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:36:44.561743 3088 tablet_bootstrap.cc:492] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: No bootstrap required, opened a new log
I20260618 22:36:44.562065 3088 ts_tablet_manager.cc:1397] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260618 22:36:44.563867 3089 tablet_bootstrap.cc:492] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: No bootstrap required, opened a new log
I20260618 22:36:44.564116 3089 ts_tablet_manager.cc:1397] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260618 22:36:44.570309 3090 tablet_bootstrap.cc:492] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: No bootstrap required, opened a new log
I20260618 22:36:44.570585 3090 ts_tablet_manager.cc:1397] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260618 22:36:44.572520 3088 raft_consensus.cc:348] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:44.572934 3088 raft_consensus.cc:374] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:36:44.573035 3088 raft_consensus.cc:729] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f54f490a53f94619a386da91da12c63c, State: Initialized, Role: FOLLOWER
I20260618 22:36:44.574234 3088 consensus_queue.cc:260] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:44.574388 3089 raft_consensus.cc:348] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:44.574800 3089 raft_consensus.cc:374] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:36:44.574900 3089 raft_consensus.cc:729] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6d9ff501c942cca6dae0246ce97c62, State: Initialized, Role: FOLLOWER
I20260618 22:36:44.575619 3089 consensus_queue.cc:260] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:44.580825 3090 raft_consensus.cc:348] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:44.581264 3090 raft_consensus.cc:374] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:36:44.581358 3090 raft_consensus.cc:729] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea081ab812f14c4dbbc7529e044f30d3, State: Initialized, Role: FOLLOWER
I20260618 22:36:44.581455 3072 heartbeater.cc:503] Master 127.18.150.190:46483 was elected leader, sending a full tablet report...
I20260618 22:36:44.582077 3090 consensus_queue.cc:260] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:44.582541 3088 ts_tablet_manager.cc:1428] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: Time spent starting tablet: real 0.020s user 0.014s sys 0.004s
I20260618 22:36:44.583596 3089 ts_tablet_manager.cc:1428] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: Time spent starting tablet: real 0.019s user 0.006s sys 0.012s
I20260618 22:36:44.583630 2809 heartbeater.cc:503] Master 127.18.150.190:46483 was elected leader, sending a full tablet report...
I20260618 22:36:44.584125 2941 heartbeater.cc:503] Master 127.18.150.190:46483 was elected leader, sending a full tablet report...
I20260618 22:36:44.584183 3090 ts_tablet_manager.cc:1428] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260618 22:36:44.705693 3073 tablet.cc:2369] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:36:44.776520 2942 tablet.cc:2369] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:36:44.819252 2810 tablet.cc:2369] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:36:44.864001 3096 raft_consensus.cc:482] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:36:44.864324 3096 raft_consensus.cc:504] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:44.866150 3096 leader_election.cc:288] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f54f490a53f94619a386da91da12c63c (127.18.150.131:38409), bd6d9ff501c942cca6dae0246ce97c62 (127.18.150.130:40131)
I20260618 22:36:44.872727 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.866692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:41153 (local address 127.18.150.130:40131)
0618 22:36:44.866887 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:44.866892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:44.867352 (+ 460us) server_negotiation.cc:407] Connection header received
0618 22:36:44.867535 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:44.867541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:44.867617 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:44.867682 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:44.868330 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.869022 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.871031 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.871846 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.871989 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.872068 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:44.872382 (+ 314us) server_negotiation.cc:299] Negotiation successful
0618 22:36:44.872498 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:36:44.872726 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.866453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:43081 (local address 127.18.150.131:38409)
0618 22:36:44.866668 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:44.866674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:44.867455 (+ 781us) server_negotiation.cc:407] Connection header received
0618 22:36:44.867602 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:44.867606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:44.867668 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:44.867776 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:36:44.868373 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.869051 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.871006 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.871846 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.871986 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.872059 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:44.872347 (+ 288us) server_negotiation.cc:299] Negotiation successful
0618 22:36:44.872481 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:36:44.873210 3097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.866393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:40131 (local address 127.18.150.129:41153)
0618 22:36:44.867216 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:44.867243 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:44.867396 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:44.867719 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:44.867724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:44.867810 (+ 86us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:44.868179 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.868189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.869175 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.869180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.870893 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.870905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.872099 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.872106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.872240 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.872261 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:36:44.872345 (+ 84us) client_negotiation.cc:240] Negotiation successful
0618 22:36:44.872493 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":30,"thread_start_us":431,"threads_started":1}
I20260618 22:36:44.873240 3098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.866463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:38409 (local address 127.18.150.129:43081)
0618 22:36:44.867339 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0618 22:36:44.867361 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:36:44.867484 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:36:44.867840 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:36:44.867844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:36:44.867872 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:36:44.868223 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.868232 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.869171 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.869175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.870870 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:36:44.870881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.872050 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.872057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:36:44.872205 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.872228 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:36:44.872299 (+ 71us) client_negotiation.cc:240] Negotiation successful
0618 22:36:44.872444 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"mutex_wait_us":89,"thread_start_us":646,"threads_started":1}
I20260618 22:36:44.874197 2896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eab06a46cf640a9bb7272c6de8cba69" candidate_uuid: "ea081ab812f14c4dbbc7529e044f30d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6d9ff501c942cca6dae0246ce97c62" is_pre_election: true
I20260618 22:36:44.874193 3027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eab06a46cf640a9bb7272c6de8cba69" candidate_uuid: "ea081ab812f14c4dbbc7529e044f30d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f54f490a53f94619a386da91da12c63c" is_pre_election: true
I20260618 22:36:44.874624 2896 raft_consensus.cc:2454] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea081ab812f14c4dbbc7529e044f30d3 in term 0.
I20260618 22:36:44.874630 3027 raft_consensus.cc:2454] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea081ab812f14c4dbbc7529e044f30d3 in term 0.
I20260618 22:36:44.875238 2697 leader_election.cc:302] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: bd6d9ff501c942cca6dae0246ce97c62, ea081ab812f14c4dbbc7529e044f30d3, f54f490a53f94619a386da91da12c63c; no voters:
I20260618 22:36:44.875612 3096 raft_consensus.cc:2781] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:36:44.875754 3096 raft_consensus.cc:482] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:36:44.875851 3096 raft_consensus.cc:3037] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:36:44.877923 3096 raft_consensus.cc:504] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:44.878572 3096 leader_election.cc:288] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [CANDIDATE]: Term 1 election: Requested vote from peers f54f490a53f94619a386da91da12c63c (127.18.150.131:38409), bd6d9ff501c942cca6dae0246ce97c62 (127.18.150.130:40131)
I20260618 22:36:44.879096 2896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eab06a46cf640a9bb7272c6de8cba69" candidate_uuid: "ea081ab812f14c4dbbc7529e044f30d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd6d9ff501c942cca6dae0246ce97c62"
I20260618 22:36:44.879096 3027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eab06a46cf640a9bb7272c6de8cba69" candidate_uuid: "ea081ab812f14c4dbbc7529e044f30d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f54f490a53f94619a386da91da12c63c"
I20260618 22:36:44.879317 2896 raft_consensus.cc:3037] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:36:44.879317 3027 raft_consensus.cc:3037] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:36:44.882498 3027 raft_consensus.cc:2454] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea081ab812f14c4dbbc7529e044f30d3 in term 1.
I20260618 22:36:44.882504 2896 raft_consensus.cc:2454] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea081ab812f14c4dbbc7529e044f30d3 in term 1.
I20260618 22:36:44.882932 2697 leader_election.cc:302] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: bd6d9ff501c942cca6dae0246ce97c62, ea081ab812f14c4dbbc7529e044f30d3, f54f490a53f94619a386da91da12c63c; no voters:
I20260618 22:36:44.883246 3096 raft_consensus.cc:2781] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:36:44.883653 3096 raft_consensus.cc:686] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 1 LEADER]: Becoming Leader. State: Replica: ea081ab812f14c4dbbc7529e044f30d3, State: Running, Role: LEADER
I20260618 22:36:44.884093 3096 consensus_queue.cc:237] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } }
I20260618 22:36:44.888662 2622 catalog_manager.cc:5495] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea081ab812f14c4dbbc7529e044f30d3 (127.18.150.129). New cstate: current_term: 1 leader_uuid: "ea081ab812f14c4dbbc7529e044f30d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea081ab812f14c4dbbc7529e044f30d3" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 38009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 } health_report { overall_health: UNKNOWN } } }
I20260618 22:36:44.986768 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:44.981332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52656 (local address 127.18.150.129:38009)
0618 22:36:44.981558 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:44.981564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:44.981584 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:36:44.981752 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:44.981756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:44.981871 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:44.982033 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:36:44.982402 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.983156 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.984577 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:44.985005 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:44.985112 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:44.985623 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:36:44.986189 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:36:44.986321 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:44.986421 (+ 100us) server_negotiation.cc:299] Negotiation successful
0618 22:36:44.986514 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260618 22:36:44.994544 3027 raft_consensus.cc:1261] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c [term 1 FOLLOWER]: Refusing update from remote peer ea081ab812f14c4dbbc7529e044f30d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:36:44.994539 2896 raft_consensus.cc:1261] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62 [term 1 FOLLOWER]: Refusing update from remote peer ea081ab812f14c4dbbc7529e044f30d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:36:44.995411 3099 consensus_queue.cc:1035] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6d9ff501c942cca6dae0246ce97c62" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 40131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:36:44.995765 3096 consensus_queue.cc:1035] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f54f490a53f94619a386da91da12c63c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:36:45.005589 3101 mvcc.cc:205] Tried to move back new op lower bound from 7298343751648296960 to 7298343751208673280. Current Snapshot: MvccSnapshot[applied={T|T < 7298343751648296960}]
I20260618 22:36:45.018503 2624 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:36:45.018735 2624 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:36:45.021970 2624 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Sending DeleteTablet for 3 replicas of tablet 3eab06a46cf640a9bb7272c6de8cba69
I20260618 22:36:45.023233 2744 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eab06a46cf640a9bb7272c6de8cba69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:36:45 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:52646
I20260618 22:36:45.023139 3007 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eab06a46cf640a9bb7272c6de8cba69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:36:45 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:55114
I20260618 22:36:45.033291 2876 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eab06a46cf640a9bb7272c6de8cba69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:36:45 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:48026
I20260618 22:36:45.034518 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 2661
I20260618 22:36:45.035152 3112 tablet_replica.cc:330] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: stopping tablet replica
I20260618 22:36:45.035993 3112 raft_consensus.cc:2229] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:36:45.037393 3112 raft_consensus.cc:2258] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:36:45.044492 3112 ts_tablet_manager.cc:1905] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:36:45.044610 3110 tablet_replica.cc:330] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: stopping tablet replica
I20260618 22:36:45.045063 3110 raft_consensus.cc:2229] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:36:45.045593 3110 raft_consensus.cc:2258] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:36:45.051101 3111 tablet_replica.cc:330] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: stopping tablet replica
I20260618 22:36:45.051609 3111 raft_consensus.cc:2229] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:36:45.052199 3112 ts_tablet_manager.cc:1918] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:36:45.052367 3112 log.cc:1196] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/3eab06a46cf640a9bb7272c6de8cba69
I20260618 22:36:45.052872 3112 ts_tablet_manager.cc:1939] T 3eab06a46cf640a9bb7272c6de8cba69 P bd6d9ff501c942cca6dae0246ce97c62: Deleting consensus metadata
I20260618 22:36:45.053045 3111 raft_consensus.cc:2258] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:36:45.054474 3110 ts_tablet_manager.cc:1905] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:36:45.054803 2610 catalog_manager.cc:4841] TS bd6d9ff501c942cca6dae0246ce97c62 (127.18.150.130:40131): tablet 3eab06a46cf640a9bb7272c6de8cba69 (table test-table [id=14e85ca4ec744489bba318b45818e4c0]) successfully deleted
I20260618 22:36:45.056336 3111 ts_tablet_manager.cc:1905] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:36:45.061585 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:45.035785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52662 (local address 127.18.150.129:38009)
0618 22:36:45.035969 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:45.035975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:45.041343 (+ 5368us) server_negotiation.cc:407] Connection header received
0618 22:36:45.041529 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:45.041535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:45.041616 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:45.041713 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:45.043771 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:45.044991 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:45.049127 (+ 4136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:45.049857 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:45.050015 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:45.054903 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:45.054915 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:45.054927 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:45.054998 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:45.057986 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:45.058888 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:45.058896 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:45.058900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:45.059082 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:45.059561 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:45.059566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:45.059570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:45.060010 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:45.060344 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:45.060934 (+ 590us) server_negotiation.cc:299] Negotiation successful
0618 22:36:45.061185 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:36:45.061698 3110 ts_tablet_manager.cc:1918] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:36:45.061851 3110 log.cc:1196] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/3eab06a46cf640a9bb7272c6de8cba69
I20260618 22:36:45.062364 3110 ts_tablet_manager.cc:1939] T 3eab06a46cf640a9bb7272c6de8cba69 P f54f490a53f94619a386da91da12c63c: Deleting consensus metadata
I20260618 22:36:45.063081 2804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:45.064203 2608 catalog_manager.cc:4841] TS f54f490a53f94619a386da91da12c63c (127.18.150.131:38409): tablet 3eab06a46cf640a9bb7272c6de8cba69 (table test-table [id=14e85ca4ec744489bba318b45818e4c0]) successfully deleted
I20260618 22:36:45.159345 3111 ts_tablet_manager.cc:1918] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:36:45.159621 3111 log.cc:1196] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/3eab06a46cf640a9bb7272c6de8cba69
I20260618 22:36:45.160172 3111 ts_tablet_manager.cc:1939] T 3eab06a46cf640a9bb7272c6de8cba69 P ea081ab812f14c4dbbc7529e044f30d3: Deleting consensus metadata
I20260618 22:36:45.161723 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 2661
I20260618 22:36:45.161793 2608 catalog_manager.cc:4841] TS ea081ab812f14c4dbbc7529e044f30d3 (127.18.150.129:38009): tablet 3eab06a46cf640a9bb7272c6de8cba69 (table test-table [id=14e85ca4ec744489bba318b45818e4c0]) successfully deleted
I20260618 22:36:45.177811 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 2813
I20260618 22:36:45.190394 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:45.178546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48042 (local address 127.18.150.130:40131)
0618 22:36:45.178691 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:45.178696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:45.178739 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:36:45.178882 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:45.178886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:45.178951 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:45.179031 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:45.180328 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:45.181074 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:45.182451 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:45.182845 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:45.182937 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:45.185309 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:45.185321 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:45.185337 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:45.185371 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:45.187866 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:45.188461 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:45.188467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:45.188469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:45.188607 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:45.188920 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:45.188927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:45.188930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:45.189326 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:45.189573 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:45.189818 (+ 245us) server_negotiation.cc:299] Negotiation successful
0618 22:36:45.190121 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:36:45.191077 2936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:45.277789 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 2813
I20260618 22:36:45.291771 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 2944
I20260618 22:36:45.303611 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:36:45.292488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55120 (local address 127.18.150.131:38409)
0618 22:36:45.292662 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:36:45.292666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:36:45.292684 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:36:45.292790 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:36:45.292794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:36:45.292867 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:36:45.292953 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:36:45.294289 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:45.294982 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:45.296175 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:36:45.296540 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:36:45.296633 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:36:45.299016 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:36:45.299030 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:36:45.299045 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:36:45.299074 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:36:45.301473 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:45.302069 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:45.302077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:45.302079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:45.302198 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:36:45.302477 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:36:45.302480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:36:45.302482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:36:45.302804 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:36:45.303007 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0618 22:36:45.303198 (+ 191us) server_negotiation.cc:299] Negotiation successful
0618 22:36:45.303365 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:36:45.304183 3067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:45.389741 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 2944
I20260618 22:36:45.402427 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 2591
I20260618 22:36:45.403509 2654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:36:45.515095 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 2591
I20260618 22:37:05.533347 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:46483
--webserver_interface=127.18.150.190
--webserver_port=46055
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:46483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:05.850919 3118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:05.851219 3118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:05.851349 3118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:05.858515 3118 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260618 22:37:05.858610 3118 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260618 22:37:05.858673 3118 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260618 22:37:05.858738 3118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:05.858809 3118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:05.858901 3118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:05.858958 3118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:05.859028 3118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:05.868685 3118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:46483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:46483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=46055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:05.869913 3118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:05.871217 3118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:05.882121 3123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:05.882153 3126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:05.882126 3124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:05.882958 3118 server_base.cc:1034] running on GCE node
I20260618 22:37:05.883721 3118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:05.885046 3118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:05.886258 3118 hybrid_clock.cc:648] HybridClock initialized: now 1781822225886207 us; error 53 us; skew 500 ppm
I20260618 22:37:05.886545 3118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:05 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822225, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:05.895222 3118 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:37:05.896668 3118 webserver.cc:460] Webserver started at http://127.18.150.190:46055/ using document root <none> and password file <none>
I20260618 22:37:05.897377 3118 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:05.897568 3118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:05.903031 3118 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:37:05.905874 3133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:05.907241 3118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:37:05.907403 3118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "c70be62673614a6ea5293395381c1f55"
format_stamp: "Formatted at 2026-06-18 22:36:20 on dist-test-slave-h7jz"
server_key: "7a41200c303f1de255d1aff511d659ef"
server_key_iv: "86d8dadc71b30d35fa9137f892ac0af4"
server_key_version: "encryptionkey@0"
I20260618 22:37:05.908365 3118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:05.939940 3118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:05.943837 3118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:05.944100 3118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:05.961129 3118 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:46483
I20260618 22:37:05.961112 3184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:46483 every 8 connection(s)
I20260618 22:37:05.962728 3118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:05.965329 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 3118
I20260618 22:37:05.966150 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:38009
--local_ip_for_outbound_sockets=127.18.150.129
--tserver_master_addrs=127.18.150.190:46483
--webserver_port=36369
--webserver_interface=127.18.150.129
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:37:05.969676 3185 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:37:05.972904 3185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Bootstrap starting.
I20260618 22:37:06.000846 3185 log.cc:824] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:06.021323 3185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:37:06.021970 3185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Bootstrap complete.
I20260618 22:37:06.032169 3185 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:37:06.033044 3185 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70be62673614a6ea5293395381c1f55, State: Initialized, Role: FOLLOWER
I20260618 22:37:06.033733 3185 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:37:06.033980 3185 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:06.034093 3185 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:06.034257 3185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 2 FOLLOWER]: Advancing to term 3
I20260618 22:37:06.037864 3185 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:37:06.038326 3185 leader_election.cc:302] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70be62673614a6ea5293395381c1f55; no voters:
I20260618 22:37:06.038932 3185 leader_election.cc:288] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260618 22:37:06.039137 3189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 3 FOLLOWER]: Leader election won for term 3
I20260618 22:37:06.040060 3189 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [term 3 LEADER]: Becoming Leader. State: Replica: c70be62673614a6ea5293395381c1f55, State: Running, Role: LEADER
I20260618 22:37:06.040769 3185 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:06.040735 3189 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } }
I20260618 22:37:06.046416 3190 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c70be62673614a6ea5293395381c1f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } } }
I20260618 22:37:06.046808 3190 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:06.047386 3191 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70be62673614a6ea5293395381c1f55. Latest consensus state: current_term: 3 leader_uuid: "c70be62673614a6ea5293395381c1f55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70be62673614a6ea5293395381c1f55" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 46483 } } }
I20260618 22:37:06.047677 3191 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:06.049749 3197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:06.060840 3197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:06.062366 3197 catalog_manager.cc:1259] Loaded cluster ID: 18df41e3270940f1a382ae3e02efea5d
I20260618 22:37:06.062515 3197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:06.066531 3197 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:06.068091 3197 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Loaded TSK: 0
I20260618 22:37:06.068306 3197 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c70be62673614a6ea5293395381c1f55: Loaded TSK: 1
F20260618 22:37:06.069650 3197 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f18cf5299ad google::LogMessage::Fail() at ??:0
@ 0x7f18cf52c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f18cf5294de google::LogMessage::Flush() at ??:0
@ 0x7f18cf52aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f18d328d7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f18d327c73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f18e4f581c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f18e4f57783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f18e4fe32a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f18e4fe30c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f18e502d5e9 std::function<>::operator()() at ??:0
@ 0x7f18e4f5bcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f18e4f59c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f18e4fe1610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f18e4fe1439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f18d2175345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f18d217891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f18d2178799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f18d2153eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f18d542d6db start_thread at ??:0
@ 0x7f18ce3c361f clone at ??:0
W20260618 22:37:06.382402 3187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:06.382663 3187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:06.382766 3187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:06.389796 3187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:06.389896 3187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:06.390058 3187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:37:06.399401 3187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:38009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=36369
--tserver_master_addrs=127.18.150.190:46483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:06.400580 3187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:06.401769 3187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:06.414280 3234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:06.414482 3236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:06.414721 3233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:06.414966 3187 server_base.cc:1034] running on GCE node
I20260618 22:37:06.415599 3187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:06.416468 3187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:06.417665 3187 hybrid_clock.cc:648] HybridClock initialized: now 1781822226417617 us; error 36 us; skew 500 ppm
I20260618 22:37:06.417986 3187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:06 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822226, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:06.420944 3187 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:37:06.422488 3187 webserver.cc:460] Webserver started at http://127.18.150.129:36369/ using document root <none> and password file <none>
I20260618 22:37:06.423161 3187 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:06.423274 3187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:06.428550 3187 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260618 22:37:06.431306 3243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:06.432672 3187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:37:06.432825 3187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "ea081ab812f14c4dbbc7529e044f30d3"
format_stamp: "Formatted at 2026-06-18 22:36:20 on dist-test-slave-h7jz"
server_key: "db13b6a1ee42a98cf25cca7493c49d93"
server_key_iv: "eaf27c76f04b3c806ff5d131d44e305e"
server_key_version: "encryptionkey@0"
I20260618 22:37:06.434017 3187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:06.475615 3187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:06.479557 3187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:06.479784 3187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:06.480718 3187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:06.482043 3187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:06.482134 3187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:06.482213 3187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:06.482265 3187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:06.511998 3187 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:38009
I20260618 22:37:06.512063 3355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:38009 every 8 connection(s)
I20260618 22:37:06.513389 3187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:37:06.518128 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:06.516186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:56317)
0618 22:37:06.516922 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:06.517213 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":261,"threads_started":1}
W20260618 22:37:06.518800 3356 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:46483 (0 consecutive failures): Network error: Failed to ping master at 127.18.150.190:46483: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
I20260618 22:37:06.519654 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:06.519309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:33847)
0618 22:37:06.519471 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:06.519543 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260618 22:37:06.520412 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:06.520112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:44699)
0618 22:37:06.520231 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:06.520305 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260618 22:37:06.520640 3356 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260618 22:37:06.522886 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 3187
I20260618 22:37:06.523516 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:40131
--local_ip_for_outbound_sockets=127.18.150.130
--tserver_master_addrs=127.18.150.190:46483
--webserver_port=36421
--webserver_interface=127.18.150.130
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:06.838711 3359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:06.838980 3359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:06.839120 3359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:06.846448 3359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:06.846549 3359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:06.846709 3359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:37:06.856251 3359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:40131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=36421
--tserver_master_addrs=127.18.150.190:46483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:06.857465 3359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:06.858763 3359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:06.870939 3364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:06.870900 3367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:06.870970 3365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:06.871861 3359 server_base.cc:1034] running on GCE node
I20260618 22:37:06.872464 3359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:06.873339 3359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:06.874536 3359 hybrid_clock.cc:648] HybridClock initialized: now 1781822226874486 us; error 32 us; skew 500 ppm
I20260618 22:37:06.874835 3359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:06 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822226, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:06.877804 3359 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:37:06.879150 3359 webserver.cc:460] Webserver started at http://127.18.150.130:36421/ using document root <none> and password file <none>
I20260618 22:37:06.879812 3359 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:06.879911 3359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:06.885133 3359 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260618 22:37:06.887871 3374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:06.889250 3359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:37:06.889405 3359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "bd6d9ff501c942cca6dae0246ce97c62"
format_stamp: "Formatted at 2026-06-18 22:36:21 on dist-test-slave-h7jz"
server_key: "8c302273fc18059616487156161de372"
server_key_iv: "a96de743e63b45906cea1aec2fe6dfa2"
server_key_version: "encryptionkey@0"
I20260618 22:37:06.890473 3359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:06.905369 3359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:06.909346 3359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:06.909587 3359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:06.910579 3359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:06.911919 3359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:06.912005 3359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:06.912117 3359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:06.912185 3359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:06.941680 3359 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:40131
I20260618 22:37:06.941740 3486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:40131 every 8 connection(s)
I20260618 22:37:06.943122 3359 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:37:06.944036 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 3359
I20260618 22:37:06.944793 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:38409
--local_ip_for_outbound_sockets=127.18.150.131
--tserver_master_addrs=127.18.150.190:46483
--webserver_port=35203
--webserver_interface=127.18.150.131
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:37:06.948537 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:06.946337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:49549)
0618 22:37:06.947152 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:06.947430 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":286,"threads_started":1}
W20260618 22:37:06.949211 3487 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:46483 (0 consecutive failures): Network error: Failed to ping master at 127.18.150.190:46483: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
I20260618 22:37:06.950076 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:06.949698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:55271)
0618 22:37:06.949853 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:06.949929 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260618 22:37:06.950839 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:06.950541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:37257)
0618 22:37:06.950659 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:06.950733 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260618 22:37:06.951048 3487 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:37:07.264093 3490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:07.264374 3490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:07.264475 3490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:07.271827 3490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:07.271943 3490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:07.272064 3490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:37:07.282094 3490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:38409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=35203
--tserver_master_addrs=127.18.150.190:46483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:07.283303 3490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:07.284523 3490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:07.296778 3495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:07.296752 3496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:07.296810 3498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:07.297967 3490 server_base.cc:1034] running on GCE node
I20260618 22:37:07.298600 3490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:07.299551 3490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:07.300748 3490 hybrid_clock.cc:648] HybridClock initialized: now 1781822227300696 us; error 71 us; skew 500 ppm
I20260618 22:37:07.301038 3490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:07 dist-test-slave-h7jz krb5kdc[2040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822227, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:07.304064 3490 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:37:07.305452 3490 webserver.cc:460] Webserver started at http://127.18.150.131:35203/ using document root <none> and password file <none>
I20260618 22:37:07.306154 3490 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:07.306267 3490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:07.311323 3490 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260618 22:37:07.313874 3505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:07.315217 3490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:37:07.315366 3490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "f54f490a53f94619a386da91da12c63c"
format_stamp: "Formatted at 2026-06-18 22:36:21 on dist-test-slave-h7jz"
server_key: "eac2db335a955438425ebfe7bfd18879"
server_key_iv: "cc8c3b2f19f111b6ffa76cafce5f697c"
server_key_version: "encryptionkey@0"
I20260618 22:37:07.316286 3490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:07.341389 3490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:07.345348 3490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:07.345593 3490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:07.346599 3490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:07.347887 3490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:07.347983 3490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:07.348086 3490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:07.348163 3490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:07.376434 3490 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:38409
I20260618 22:37:07.376500 3617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:38409 every 8 connection(s)
I20260618 22:37:07.377820 3490 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:37:07.383128 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:07.380890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:45343)
0618 22:37:07.381705 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:07.382033 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":305,"threads_started":1}
W20260618 22:37:07.383738 3618 heartbeater.cc:644] Failed to heartbeat to 127.18.150.190:46483 (0 consecutive failures): Network error: Failed to ping master at 127.18.150.190:46483: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
I20260618 22:37:07.384582 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:07.384254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:44125)
0618 22:37:07.384415 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:07.384483 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260618 22:37:07.385473 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:07.385116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:34041)
0618 22:37:07.385262 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:07.385345 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260618 22:37:07.385660 3618 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260618 22:37:07.387022 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 3490
I20260618 22:37:07.522082 3622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:07.521231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:58425)
0618 22:37:07.521728 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:07.521834 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":238,"threads_started":1}
I20260618 22:37:07.952328 3623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:07.951574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:53211)
0618 22:37:07.952010 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:07.952114 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":219,"threads_started":1}
I20260618 22:37:08.387095 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:08.386269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:36115)
0618 22:37:08.386752 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:08.386842 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":253,"threads_started":1}
I20260618 22:37:08.523627 3625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:08.522922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:58947)
0618 22:37:08.523375 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:08.523460 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":232,"threads_started":1}
I20260618 22:37:08.953929 3626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:08.953225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:39397)
0618 22:37:08.953665 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:08.953757 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":214,"threads_started":1}
I20260618 22:37:09.388780 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:09.388004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:41031)
0618 22:37:09.388503 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:09.388598 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":236,"threads_started":1}
I20260618 22:37:09.525236 3628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:09.524495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:54489)
0618 22:37:09.524967 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:09.525046 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":260,"threads_started":1}
I20260618 22:37:09.955519 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:09.954780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:49531)
0618 22:37:09.955268 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:09.955348 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":262,"threads_started":1}
I20260618 22:37:10.390301 3630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:10.389585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:33851)
0618 22:37:10.390040 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:10.390122 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":233,"threads_started":1}
I20260618 22:37:10.526888 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:10.526191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:45333)
0618 22:37:10.526635 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:10.526720 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":223,"threads_started":1}
I20260618 22:37:10.957120 3632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:10.956379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:59015)
0618 22:37:10.956791 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:10.956876 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":191,"threads_started":1}
I20260618 22:37:11.391868 3633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:11.391075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:43999)
0618 22:37:11.391579 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:11.391670 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":256,"threads_started":1}
I20260618 22:37:11.528419 3634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:11.527725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:42589)
0618 22:37:11.528173 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:11.528258 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":243,"threads_started":1}
I20260618 22:37:11.958719 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:11.958029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:43997)
0618 22:37:11.958465 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:11.958544 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":218,"threads_started":1}
I20260618 22:37:12.393555 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:12.392755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:48865)
0618 22:37:12.393255 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:12.393346 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":240,"threads_started":1}
I20260618 22:37:12.530047 3637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:12.529283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:40699)
0618 22:37:12.529744 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:12.529837 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":236,"threads_started":1}
I20260618 22:37:12.960191 3638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:12.959506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:59315)
0618 22:37:12.959935 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:12.960024 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":228,"threads_started":1}
I20260618 22:37:13.395221 3639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:13.394488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:36371)
0618 22:37:13.394958 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:13.395054 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":236,"threads_started":1}
I20260618 22:37:13.531703 3640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:13.530949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:37485)
0618 22:37:13.531401 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:13.531481 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":194,"threads_started":1}
I20260618 22:37:13.961992 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:13.961093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:37127)
0618 22:37:13.961600 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:13.961683 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":227,"threads_started":1}
I20260618 22:37:14.396858 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:14.396075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:47103)
0618 22:37:14.396549 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:14.396633 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":208,"threads_started":1}
I20260618 22:37:14.533385 3643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:14.532645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:51053)
0618 22:37:14.533082 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:14.533224 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":218,"threads_started":1}
I20260618 22:37:14.963498 3644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:14.962835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:47749)
0618 22:37:14.963258 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:14.963341 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":208,"threads_started":1}
I20260618 22:37:15.398372 3645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:15.397673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:45089)
0618 22:37:15.398113 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:15.398197 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":206,"threads_started":1}
I20260618 22:37:15.535225 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:15.534326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:51627)
0618 22:37:15.534899 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:15.535037 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":268,"threads_started":1}
I20260618 22:37:15.965116 3647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:15.964367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:49101)
0618 22:37:15.964860 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:15.964946 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":233,"threads_started":1}
I20260618 22:37:16.400079 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:16.399224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:60035)
0618 22:37:16.399786 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:16.399892 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":243,"threads_started":1}
I20260618 22:37:16.536826 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:16.536103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:54579)
0618 22:37:16.536552 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:16.536667 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":231,"threads_started":1}
I20260618 22:37:16.966809 3650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:16.966063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:37675)
0618 22:37:16.966523 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:16.966617 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":239,"threads_started":1}
I20260618 22:37:17.401682 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:17.400987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:46671)
0618 22:37:17.401429 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:17.401524 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":243,"threads_started":1}
I20260618 22:37:17.538422 3652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:17.537632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:39817)
0618 22:37:17.538123 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:17.538219 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":229,"threads_started":1}
I20260618 22:37:17.968518 3653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:17.967769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:35787)
0618 22:37:17.968235 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:17.968335 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":218,"threads_started":1}
I20260618 22:37:18.403211 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:18.402519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:34939)
0618 22:37:18.402935 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:18.403027 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":210,"threads_started":1}
I20260618 22:37:18.539973 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:18.539270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:49617)
0618 22:37:18.539732 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:18.539817 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":235,"threads_started":1}
I20260618 22:37:18.970152 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:18.969434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:58159)
0618 22:37:18.969876 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:18.969990 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":212,"threads_started":1}
I20260618 22:37:19.404729 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:19.403995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:58759)
0618 22:37:19.404493 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:19.404575 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":247,"threads_started":1}
I20260618 22:37:19.541501 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:19.540785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:50871)
0618 22:37:19.541211 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:19.541305 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":209,"threads_started":1}
I20260618 22:37:19.971784 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:19.970993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:43985)
0618 22:37:19.971495 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:19.971596 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":249,"threads_started":1}
I20260618 22:37:20.406390 3660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:20.405614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:52887)
0618 22:37:20.406112 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:20.406210 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":220,"threads_started":1}
I20260618 22:37:20.543059 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:20.542386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:41781)
0618 22:37:20.542794 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:20.542883 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":200,"threads_started":1}
I20260618 22:37:20.973583 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:20.972695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:43891)
0618 22:37:20.973250 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:20.973365 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":272,"threads_started":1}
I20260618 22:37:21.407898 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:21.407184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:39625)
0618 22:37:21.407625 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:21.407718 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":225,"threads_started":1}
I20260618 22:37:21.544931 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:21.544169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.129:51649)
0618 22:37:21.544622 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:21.544724 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":233,"threads_started":1}
I20260618 22:37:21.975395 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:21.974700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.130:44029)
0618 22:37:21.975122 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:21.975218 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":209,"threads_started":1}
I20260618 22:37:22.389822 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 3187
I20260618 22:37:22.409971 3667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:22.409084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:46483 (local address 127.18.150.131:53701)
0618 22:37:22.409539 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:22.409697 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.190:46483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":523,"threads_started":1}
I20260618 22:37:22.413411 3666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:22.391111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.18.150.129:38009)
0618 22:37:22.391662 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:22.391676 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:22.391711 (+ 35us) server_negotiation.cc:407] Connection header received
0618 22:37:22.393167 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:22.393177 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:22.393606 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:22.394005 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:22.395495 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:22.396726 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:22.397963 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:22.398666 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:22.398842 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:22.404812 (+ 5970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:22.404824 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:22.404850 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:22.404903 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:22.408831 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:22.410795 (+ 1964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:22.410805 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:22.410815 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:22.411000 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:22.411379 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:22.411385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:22.411388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:22.411931 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:22.412233 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:22.412695 (+ 462us) server_negotiation.cc:299] Negotiation successful
0618 22:37:22.413139 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":240,"threads_started":1}
I20260618 22:37:22.414777 3351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:22.497670 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 3187
I20260618 22:37:22.511437 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 3359
I20260618 22:37:22.528694 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:22.512391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.18.150.130:40131)
0618 22:37:22.512940 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:22.512954 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:22.512992 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:37:22.514575 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:22.514586 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:22.515036 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:22.515332 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:22.516601 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:22.517852 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:22.519201 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:22.520014 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:22.520145 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:22.522730 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:22.522738 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:22.522753 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:22.522790 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:22.525543 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:22.526395 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:22.526402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:22.526408 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:22.526570 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:22.526893 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:22.526897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:22.526898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:22.527393 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:22.527610 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:22.528015 (+ 405us) server_negotiation.cc:299] Negotiation successful
0618 22:37:22.528423 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":270,"threads_started":1}
I20260618 22:37:22.530081 3482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:22.613771 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 3359
I20260618 22:37:22.628222 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 3490
I20260618 22:37:22.645731 3671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:22.629194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.18.150.131:38409)
0618 22:37:22.629761 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:22.629776 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:22.629818 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:37:22.631439 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:22.631449 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:22.631882 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:22.632206 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:22.633599 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:22.634852 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:22.636076 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:22.636739 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:22.636921 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:22.639512 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:22.639520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:22.639536 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:22.639580 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:22.642541 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:22.643216 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:22.643223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:22.643229 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:22.643399 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:22.643776 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:22.643780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:22.643782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:22.644296 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:22.644563 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:22.645025 (+ 462us) server_negotiation.cc:299] Negotiation successful
0618 22:37:22.645440 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":249,"threads_started":1}
I20260618 22:37:22.647056 3613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:22.728799 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 3490
2026-06-18T22:37:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (63983 ms)
[----------] 30 tests from SecurityITest (281800 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:22 dist-test-slave-h7jz krb5kdc[3675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:22 dist-test-slave-h7jz krb5kdc[3675](info): set up 2 sockets
Jun 18 22:37:22 dist-test-slave-h7jz krb5kdc[3675](info): commencing operation
krb5kdc: starting...
W20260618 22:37:23.734095 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.970s user 0.005s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:23 dist-test-slave-h7jz krb5kdc[3675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:37:23.957438 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:40895
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:40895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:24.272719 3688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:24.273023 3688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:24.273154 3688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:24.280453 3688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:24.280565 3688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:24.280617 3688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:24.280695 3688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:24.280748 3688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:24.290762 3688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:40895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:40895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:24.292069 3688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:24.293370 3688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:24.305358 3696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:24.305516 3688 server_base.cc:1034] running on GCE node
W20260618 22:37:24.305167 3694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:24.305204 3693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:24.306638 3688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:24.308019 3688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:24.309223 3688 hybrid_clock.cc:648] HybridClock initialized: now 1781822244309160 us; error 45 us; skew 500 ppm
I20260618 22:37:24.309547 3688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:24 dist-test-slave-h7jz krb5kdc[3675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822244, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:24.312528 3688 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:37:24.313958 3688 webserver.cc:460] Webserver started at http://127.18.150.190:34415/ using document root <none> and password file <none>
I20260618 22:37:24.314613 3688 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:24.314720 3688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:24.314970 3688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:24.317842 3688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "f206b86ff29048f38d60fa5d22d9b0e3"
format_stamp: "Formatted at 2026-06-18 22:37:24 on dist-test-slave-h7jz"
server_key: "54ef39f48b1b3b54405be93dc240891a"
server_key_iv: "271248c551250276e0cc9049b7b5eed7"
server_key_version: "encryptionkey@0"
I20260618 22:37:24.318575 3688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "f206b86ff29048f38d60fa5d22d9b0e3"
format_stamp: "Formatted at 2026-06-18 22:37:24 on dist-test-slave-h7jz"
server_key: "54ef39f48b1b3b54405be93dc240891a"
server_key_iv: "271248c551250276e0cc9049b7b5eed7"
server_key_version: "encryptionkey@0"
I20260618 22:37:24.323506 3688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:37:24.326756 3703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:24.328052 3688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:37:24.328203 3688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "f206b86ff29048f38d60fa5d22d9b0e3"
format_stamp: "Formatted at 2026-06-18 22:37:24 on dist-test-slave-h7jz"
server_key: "54ef39f48b1b3b54405be93dc240891a"
server_key_iv: "271248c551250276e0cc9049b7b5eed7"
server_key_version: "encryptionkey@0"
I20260618 22:37:24.328382 3688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:24.363890 3688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:24.367805 3688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:24.368060 3688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:24.384624 3688 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:40895
I20260618 22:37:24.384630 3754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:40895 every 8 connection(s)
I20260618 22:37:24.385983 3688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:24.389207 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 3688
I20260618 22:37:24.389427 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:24.389991 3755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:24.390282 19034 external_mini_cluster.cc:1428] Setting key 7ec513dea131117e6a71c317e86aa330
I20260618 22:37:24.404105 3755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3: Bootstrap starting.
Jun 18 22:37:24 dist-test-slave-h7jz krb5kdc[3675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:24.407158 3755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:24.408123 3755 log.cc:824] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:24.410697 3755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3: No bootstrap required, opened a new log
I20260618 22:37:24.412609 3758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:24.392254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60666 (local address 127.18.150.190:40895)
0618 22:37:24.393070 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:24.393081 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:24.393131 (+ 50us) server_negotiation.cc:407] Connection header received
0618 22:37:24.395015 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:24.395038 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:24.395415 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:24.395743 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:24.397406 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:24.398709 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:24.400598 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:24.401294 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:24.401424 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:24.405928 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:24.405966 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:24.405986 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:24.406024 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:24.408897 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:24.409527 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:24.409533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:24.409539 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:24.409692 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:24.410077 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:24.410080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:24.410082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:24.410566 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:24.410843 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:24.411281 (+ 438us) server_negotiation.cc:299] Negotiation successful
0618 22:37:24.411665 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":339,"threads_started":1}
I20260618 22:37:24.418507 3755 raft_consensus.cc:348] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f206b86ff29048f38d60fa5d22d9b0e3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40895 } }
I20260618 22:37:24.418867 3755 raft_consensus.cc:374] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:24.418958 3755 raft_consensus.cc:729] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f206b86ff29048f38d60fa5d22d9b0e3, State: Initialized, Role: FOLLOWER
I20260618 22:37:24.419487 3755 consensus_queue.cc:260] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f206b86ff29048f38d60fa5d22d9b0e3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40895 } }
I20260618 22:37:24.419690 3755 raft_consensus.cc:388] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:24.419816 3755 raft_consensus.cc:482] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:24.419961 3755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:24.422230 3755 raft_consensus.cc:504] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f206b86ff29048f38d60fa5d22d9b0e3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40895 } }
I20260618 22:37:24.422676 3755 leader_election.cc:302] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f206b86ff29048f38d60fa5d22d9b0e3; no voters:
I20260618 22:37:24.423266 3755 leader_election.cc:288] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:24.423362 3760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:24.423935 3760 raft_consensus.cc:686] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [term 1 LEADER]: Becoming Leader. State: Replica: f206b86ff29048f38d60fa5d22d9b0e3, State: Running, Role: LEADER
I20260618 22:37:24.424415 3760 consensus_queue.cc:237] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f206b86ff29048f38d60fa5d22d9b0e3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40895 } }
I20260618 22:37:24.425107 3755 sys_catalog.cc:564] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:24.426944 3761 sys_catalog.cc:455] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f206b86ff29048f38d60fa5d22d9b0e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f206b86ff29048f38d60fa5d22d9b0e3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40895 } } }
I20260618 22:37:24.426915 3762 sys_catalog.cc:455] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f206b86ff29048f38d60fa5d22d9b0e3. Latest consensus state: current_term: 1 leader_uuid: "f206b86ff29048f38d60fa5d22d9b0e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f206b86ff29048f38d60fa5d22d9b0e3" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 40895 } } }
I20260618 22:37:24.427397 3762 sys_catalog.cc:458] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:24.427397 3761 sys_catalog.cc:458] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:24.429761 3766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:24.434791 3766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:24.443462 3766 catalog_manager.cc:1347] Generated new cluster ID: 66641216c9e04f20aaa549d333cf64f6
I20260618 22:37:24.443557 3766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:24.463035 3766 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:24.464310 3766 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:24.473380 3766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3: Generated new TSK 0
I20260618 22:37:24.474201 3766 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:37:24.539335 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40895
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:24.866210 3781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:24.866494 3781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:24.866638 3781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:24.874114 3781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:24.874233 3781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:24.874393 3781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:37:24.884176 3781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:24.885491 3781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:24.886880 3781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:24.899502 3786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:24.899523 3789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:24.899479 3787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:24.900440 3781 server_base.cc:1034] running on GCE node
I20260618 22:37:24.901118 3781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:24.902077 3781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:24.903296 3781 hybrid_clock.cc:648] HybridClock initialized: now 1781822244903234 us; error 46 us; skew 500 ppm
I20260618 22:37:24.903611 3781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:24 dist-test-slave-h7jz krb5kdc[3675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822244, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:24.906831 3781 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:37:24.908318 3781 webserver.cc:460] Webserver started at http://127.18.150.129:37843/ using document root <none> and password file <none>
I20260618 22:37:24.908986 3781 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:24.909117 3781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:24.909425 3781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:24.912569 3781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "1f79f479389f4d4c8eac640c66fede48"
format_stamp: "Formatted at 2026-06-18 22:37:24 on dist-test-slave-h7jz"
server_key: "30f4c7a58ed444c7626a87d89444512a"
server_key_iv: "631a4a1a1748ba3992f9615dfcb0ab6c"
server_key_version: "encryptionkey@0"
I20260618 22:37:24.913367 3781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "1f79f479389f4d4c8eac640c66fede48"
format_stamp: "Formatted at 2026-06-18 22:37:24 on dist-test-slave-h7jz"
server_key: "30f4c7a58ed444c7626a87d89444512a"
server_key_iv: "631a4a1a1748ba3992f9615dfcb0ab6c"
server_key_version: "encryptionkey@0"
I20260618 22:37:24.918578 3781 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260618 22:37:24.922078 3796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:24.923429 3781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:37:24.923586 3781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "1f79f479389f4d4c8eac640c66fede48"
format_stamp: "Formatted at 2026-06-18 22:37:24 on dist-test-slave-h7jz"
server_key: "30f4c7a58ed444c7626a87d89444512a"
server_key_iv: "631a4a1a1748ba3992f9615dfcb0ab6c"
server_key_version: "encryptionkey@0"
I20260618 22:37:24.923780 3781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:24.970424 3781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:24.974934 3781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:24.975203 3781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:24.976191 3781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:24.977481 3781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:24.977550 3781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:24.977628 3781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:24.977672 3781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:25.005769 3781 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:35829
I20260618 22:37:25.005823 3908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:35829 every 8 connection(s)
I20260618 22:37:25.007082 3781 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:37:25.015295 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 3781
I20260618 22:37:25.015475 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:37:25.016052 19034 external_mini_cluster.cc:1428] Setting key 1adeed8fa4fe6eed4840adf2be6e7b00
Jun 18 22:37:25 dist-test-slave-h7jz krb5kdc[3675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822244, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:25.026440 3911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:25.009493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:55519 (local address 127.18.150.190:40895)
0618 22:37:25.009956 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:25.009963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:25.010877 (+ 914us) server_negotiation.cc:407] Connection header received
0618 22:37:25.012617 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:25.012621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:25.012687 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:25.012779 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:25.014307 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.015005 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.016603 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.017027 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.017159 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:25.020912 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:25.020917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:25.020920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:25.020951 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:25.023341 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:25.024112 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:25.024115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:25.024117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:25.024227 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:25.024661 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:25.024663 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:25.024665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:25.024923 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:25.025029 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:25.025813 (+ 784us) server_negotiation.cc:299] Negotiation successful
0618 22:37:25.026066 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":216,"threads_started":1}
I20260618 22:37:25.027437 3912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:25.010078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:40895 (local address 127.18.150.129:55519)
0618 22:37:25.010729 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:25.010766 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:25.012372 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:25.013010 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:25.013018 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:25.013538 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:25.014154 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:25.014174 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.015141 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.015145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:25.016469 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:25.016481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.017252 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.017257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:25.017446 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:25.018160 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:37:25.018190 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:25.020707 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:25.023463 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:25.023471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:25.023485 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:25.024004 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:25.024324 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:25.024329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:25.024331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:25.024544 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:25.025034 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:25.025041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:25.025445 (+ 404us) client_negotiation.cc:769] Sending connection context
0618 22:37:25.025684 (+ 239us) client_negotiation.cc:240] Negotiation successful
0618 22:37:25.026213 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":249,"threads_started":1}
I20260618 22:37:25.028919 3909 heartbeater.cc:347] Connected to a master server at 127.18.150.190:40895
I20260618 22:37:25.029228 3909 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:25.029974 3909 heartbeater.cc:511] Master 127.18.150.190:40895 requested a full tablet report, sending...
I20260618 22:37:25.032160 3720 ts_manager.cc:194] Registered new tserver with Master: 1f79f479389f4d4c8eac640c66fede48 (127.18.150.129:35829)
I20260618 22:37:25.033720 3720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:55519
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:37:25.066833 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40895
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:25.376134 3915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:25.376386 3915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:25.376518 3915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:25.383518 3915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:25.383613 3915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:25.383754 3915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:37:25.393177 3915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:25.394379 3915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:25.395604 3915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:25.407533 3920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:25.407594 3921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:25.407632 3923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:25.408622 3915 server_base.cc:1034] running on GCE node
I20260618 22:37:25.409184 3915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:25.410061 3915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:25.411252 3915 hybrid_clock.cc:648] HybridClock initialized: now 1781822245411200 us; error 46 us; skew 500 ppm
I20260618 22:37:25.411530 3915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:25 dist-test-slave-h7jz krb5kdc[3675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822245, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:25.414378 3915 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:37:25.415668 3915 webserver.cc:460] Webserver started at http://127.18.150.130:34101/ using document root <none> and password file <none>
I20260618 22:37:25.416352 3915 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:25.416464 3915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:25.416760 3915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:25.419543 3915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "f3d963409b0f48a4be60dc261fcd1752"
format_stamp: "Formatted at 2026-06-18 22:37:25 on dist-test-slave-h7jz"
server_key: "3e79242b4e4cb9a5ab6d1fa8820ef1e5"
server_key_iv: "133860eb2117fdc5e3dedaa88dda5d54"
server_key_version: "encryptionkey@0"
I20260618 22:37:25.420244 3915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "f3d963409b0f48a4be60dc261fcd1752"
format_stamp: "Formatted at 2026-06-18 22:37:25 on dist-test-slave-h7jz"
server_key: "3e79242b4e4cb9a5ab6d1fa8820ef1e5"
server_key_iv: "133860eb2117fdc5e3dedaa88dda5d54"
server_key_version: "encryptionkey@0"
I20260618 22:37:25.425000 3915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260618 22:37:25.428341 3930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:25.429605 3915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260618 22:37:25.429757 3915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "f3d963409b0f48a4be60dc261fcd1752"
format_stamp: "Formatted at 2026-06-18 22:37:25 on dist-test-slave-h7jz"
server_key: "3e79242b4e4cb9a5ab6d1fa8820ef1e5"
server_key_iv: "133860eb2117fdc5e3dedaa88dda5d54"
server_key_version: "encryptionkey@0"
I20260618 22:37:25.429934 3915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:25.478845 3915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:25.482782 3915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:25.483037 3915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:25.483956 3915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:25.485242 3915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:25.485318 3915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:25.485440 3915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:25.485491 3915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:25.514370 3915 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:44789
I20260618 22:37:25.514422 4042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:44789 every 8 connection(s)
I20260618 22:37:25.515672 3915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:37:25.518064 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 3915
I20260618 22:37:25.518260 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:37:25.518772 19034 external_mini_cluster.cc:1428] Setting key 14530e016466938f81473582a824dbcf
Jun 18 22:37:25 dist-test-slave-h7jz krb5kdc[3675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822245, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:25.535689 3911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:25.518058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:39933 (local address 127.18.150.190:40895)
0618 22:37:25.518228 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:25.518235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:25.519710 (+ 1475us) server_negotiation.cc:407] Connection header received
0618 22:37:25.521804 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:25.521810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:25.521886 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:25.521985 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:25.523417 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.524100 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.525673 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.526127 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.526232 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:25.529657 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:25.529663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:25.529668 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:25.529704 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:25.532640 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:25.533406 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:25.533411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:25.533415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:25.533590 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:25.534034 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:25.534038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:25.534042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:25.534377 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:25.534528 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:25.535148 (+ 620us) server_negotiation.cc:299] Negotiation successful
0618 22:37:25.535378 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:37:25.536492 4045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:25.518774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:40895 (local address 127.18.150.130:39933)
0618 22:37:25.519554 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:25.519607 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:25.521551 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:25.522221 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:25.522229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:25.522716 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:25.523262 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:25.523280 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.524236 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.524240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:25.525536 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:25.525564 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.526348 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.526357 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:25.526584 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:25.527205 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:37:25.527230 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:25.529452 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:25.532768 (+ 3316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:25.532774 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:25.532786 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:25.533291 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:25.533695 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:25.533697 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:25.533700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:25.533922 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:25.534483 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:25.534489 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:25.534810 (+ 321us) client_negotiation.cc:769] Sending connection context
0618 22:37:25.535022 (+ 212us) client_negotiation.cc:240] Negotiation successful
0618 22:37:25.535407 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":286,"threads_started":1}
I20260618 22:37:25.537868 4043 heartbeater.cc:347] Connected to a master server at 127.18.150.190:40895
I20260618 22:37:25.538210 4043 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:25.539060 4043 heartbeater.cc:511] Master 127.18.150.190:40895 requested a full tablet report, sending...
I20260618 22:37:25.540627 3720 ts_manager.cc:194] Registered new tserver with Master: f3d963409b0f48a4be60dc261fcd1752 (127.18.150.130:44789)
I20260618 22:37:25.541446 3720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:39933
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:37:25.569259 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40895
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:25.881034 4048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:25.881309 4048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:25.881429 4048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:25.888700 4048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:25.888804 4048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:25.888945 4048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:37:25.898689 4048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:40895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:25.899915 4048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:25.901213 4048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:25.913918 4053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:25.913888 4056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:25.913904 4054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:25.914743 4048 server_base.cc:1034] running on GCE node
I20260618 22:37:25.915313 4048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:25.916134 4048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:25.917337 4048 hybrid_clock.cc:648] HybridClock initialized: now 1781822245917291 us; error 39 us; skew 500 ppm
I20260618 22:37:25.917609 4048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:25 dist-test-slave-h7jz krb5kdc[3675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822245, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:25.920476 4048 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:37:25.921801 4048 webserver.cc:460] Webserver started at http://127.18.150.131:34263/ using document root <none> and password file <none>
I20260618 22:37:25.922463 4048 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:25.922575 4048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:25.922850 4048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:25.925582 4048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "0e0b2516e64c4b619e0ad44af776cd45"
format_stamp: "Formatted at 2026-06-18 22:37:25 on dist-test-slave-h7jz"
server_key: "d4745bc41d07752fc205fd9431fdd1ca"
server_key_iv: "e96559108c60273cb62b2e98d728b3e0"
server_key_version: "encryptionkey@0"
I20260618 22:37:25.926368 4048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "0e0b2516e64c4b619e0ad44af776cd45"
format_stamp: "Formatted at 2026-06-18 22:37:25 on dist-test-slave-h7jz"
server_key: "d4745bc41d07752fc205fd9431fdd1ca"
server_key_iv: "e96559108c60273cb62b2e98d728b3e0"
server_key_version: "encryptionkey@0"
I20260618 22:37:25.931674 4048 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:37:25.934965 4063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:25.936276 4048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:37:25.936417 4048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "0e0b2516e64c4b619e0ad44af776cd45"
format_stamp: "Formatted at 2026-06-18 22:37:25 on dist-test-slave-h7jz"
server_key: "d4745bc41d07752fc205fd9431fdd1ca"
server_key_iv: "e96559108c60273cb62b2e98d728b3e0"
server_key_version: "encryptionkey@0"
I20260618 22:37:25.936589 4048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:25.952947 4048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:25.956614 4048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:25.956868 4048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:25.957760 4048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:25.959046 4048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:25.959120 4048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:25.959239 4048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:25.959288 4048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:25.987984 4048 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:45743
I20260618 22:37:25.988040 4175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:45743 every 8 connection(s)
I20260618 22:37:25.989181 4048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:37:25.989847 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 4048
I20260618 22:37:25.990048 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:37:25.990582 19034 external_mini_cluster.cc:1428] Setting key fe5e71ee372d5f05e82fd7be1bd7fbe0
Jun 18 22:37:25 dist-test-slave-h7jz krb5kdc[3675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822245, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:26.008721 3911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:25.991668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:45103 (local address 127.18.150.190:40895)
0618 22:37:25.991882 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:25.991888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:25.993088 (+ 1200us) server_negotiation.cc:407] Connection header received
0618 22:37:25.995049 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:25.995053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:25.995161 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:25.995247 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:25.996753 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.997470 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.999000 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.999406 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.999513 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.002661 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:26.002665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:26.002668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:26.002705 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:26.005854 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:26.006521 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:26.006526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:26.006529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:26.006693 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:26.007050 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:26.007055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:26.007057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:26.007387 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:26.007534 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:26.008132 (+ 598us) server_negotiation.cc:299] Negotiation successful
0618 22:37:26.008371 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:37:26.009438 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:25.992253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:40895 (local address 127.18.150.131:45103)
0618 22:37:25.992951 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:25.992992 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:25.994802 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:25.995507 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:25.995515 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:25.996023 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:25.996593 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:25.996611 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.997618 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.997622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:25.998871 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:25.998883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:25.999611 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:25.999618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:25.999836 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.000392 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:37:26.000413 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:26.002487 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:26.005990 (+ 3503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:26.005999 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:26.006014 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:26.006399 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:26.006786 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:26.006789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:26.006791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:26.006955 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:26.007490 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:26.007494 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:26.007790 (+ 296us) client_negotiation.cc:769] Sending connection context
0618 22:37:26.007992 (+ 202us) client_negotiation.cc:240] Negotiation successful
0618 22:37:26.008335 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":288,"threads_started":1}
I20260618 22:37:26.010644 4176 heartbeater.cc:347] Connected to a master server at 127.18.150.190:40895
I20260618 22:37:26.010922 4176 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:26.011546 4176 heartbeater.cc:511] Master 127.18.150.190:40895 requested a full tablet report, sending...
I20260618 22:37:26.012827 3720 ts_manager.cc:194] Registered new tserver with Master: 0e0b2516e64c4b619e0ad44af776cd45 (127.18.150.131:45743)
I20260618 22:37:26.013586 3720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:45103
I20260618 22:37:26.017824 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:37:26.034199 3911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.022089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60682 (local address 127.18.150.190:40895)
0618 22:37:26.022289 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:26.022294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:26.022681 (+ 387us) server_negotiation.cc:407] Connection header received
0618 22:37:26.022827 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:26.022831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:26.022887 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:26.022954 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:26.024347 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.025129 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.026609 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.026994 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.027085 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.029403 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:26.029408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:26.029411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:26.029440 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:26.031887 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:26.032537 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:26.032540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:26.032543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:26.032658 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:26.032956 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:26.032959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:26.032961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:26.033316 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:26.033484 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:26.033737 (+ 253us) server_negotiation.cc:299] Negotiation successful
0618 22:37:26.033883 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260618 22:37:26.037242 3909 heartbeater.cc:503] Master 127.18.150.190:40895 was elected leader, sending a full tablet report...
I20260618 22:37:26.037912 3720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:37:26.044621 3720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:37:26.076855 4189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.055672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:35829 (local address 127.0.0.1:57796)
0618 22:37:26.067927 (+ 12255us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:26.067966 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:26.068152 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:26.069878 (+ 1726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:26.069885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:26.069911 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:26.070342 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:26.070357 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.072417 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.072423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:26.074501 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:26.074513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.075844 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.075850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:26.076053 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.076108 (+ 55us) client_negotiation.cc:769] Sending connection context
0618 22:37:26.076234 (+ 126us) client_negotiation.cc:240] Negotiation successful
0618 22:37:26.076437 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12001,"mutex_wait_us":1,"thread_start_us":14930,"threads_started":1}
I20260618 22:37:26.078125 4190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.055922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57796 (local address 127.18.150.129:35829)
0618 22:37:26.068836 (+ 12914us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:26.068846 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:26.068877 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:37:26.069002 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:26.069012 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:26.069293 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:26.069492 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:26.070869 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.072295 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.074650 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.075650 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.075837 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.076028 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:26.077594 (+ 1566us) server_negotiation.cc:299] Negotiation successful
0618 22:37:26.077770 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12732,"thread_start_us":12582,"threads_started":1}
I20260618 22:37:26.078480 4191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.055889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:45743 (local address 127.0.0.1:45908)
0618 22:37:26.069579 (+ 13690us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:26.069596 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:26.069713 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:26.071434 (+ 1721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:26.071439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:26.071460 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:26.071852 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:26.071863 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.073630 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.073635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:26.075746 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:26.075762 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.077739 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.077745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:26.077916 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.077967 (+ 51us) client_negotiation.cc:769] Sending connection context
0618 22:37:26.078028 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:37:26.078148 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13562,"thread_start_us":11650,"threads_started":1}
I20260618 22:37:26.079242 4193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.056810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53734 (local address 127.18.150.130:44789)
0618 22:37:26.069171 (+ 12361us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:26.069180 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:26.070984 (+ 1804us) server_negotiation.cc:407] Connection header received
0618 22:37:26.071120 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:26.071128 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:26.071368 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:26.071549 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:26.072248 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.073725 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.076145 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.077895 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.078069 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.078238 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:26.078764 (+ 526us) server_negotiation.cc:299] Negotiation successful
0618 22:37:26.078920 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12199,"thread_start_us":12064,"threads_started":1}
I20260618 22:37:26.079382 4192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.056436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45908 (local address 127.18.150.131:45743)
0618 22:37:26.070933 (+ 14497us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:26.070941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:26.070971 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:37:26.071067 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:26.071074 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:26.071270 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:26.071433 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:26.072052 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.073459 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.075941 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.077529 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.078758 (+ 1229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.078913 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:26.078964 (+ 51us) server_negotiation.cc:299] Negotiation successful
0618 22:37:26.079069 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14342,"thread_start_us":14188,"threads_started":1}
I20260618 22:37:26.079813 4188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.056246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:44789 (local address 127.0.0.1:53734)
0618 22:37:26.070848 (+ 14602us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:26.070865 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:26.070983 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:26.071708 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:26.071713 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:26.071737 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:26.072094 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:26.072103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.073865 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.073870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:26.075950 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:26.075962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.078389 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.078395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:26.078562 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.078596 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:37:26.079361 (+ 765us) client_negotiation.cc:240] Negotiation successful
0618 22:37:26.079495 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14490,"mutex_wait_us":1,"thread_start_us":13083,"threads_started":1}
I20260618 22:37:26.082254 3844 tablet_service.cc:1467] Processing CreateTablet for tablet d43849cde5dc462eb6d299194501ce93 (DEFAULT_TABLE table=test-table [id=ff938b7526eb430494c1a9145fcf87ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:26.082588 4111 tablet_service.cc:1467] Processing CreateTablet for tablet d43849cde5dc462eb6d299194501ce93 (DEFAULT_TABLE table=test-table [id=ff938b7526eb430494c1a9145fcf87ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:26.083125 3978 tablet_service.cc:1467] Processing CreateTablet for tablet d43849cde5dc462eb6d299194501ce93 (DEFAULT_TABLE table=test-table [id=ff938b7526eb430494c1a9145fcf87ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:26.083917 4111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43849cde5dc462eb6d299194501ce93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:26.084106 3844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43849cde5dc462eb6d299194501ce93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:26.084347 3978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43849cde5dc462eb6d299194501ce93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:26.095695 4194 tablet_bootstrap.cc:492] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: Bootstrap starting.
I20260618 22:37:26.098202 4195 tablet_bootstrap.cc:492] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: Bootstrap starting.
I20260618 22:37:26.099030 4194 tablet_bootstrap.cc:654] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:26.099538 4196 tablet_bootstrap.cc:492] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: Bootstrap starting.
I20260618 22:37:26.100385 4194 log.cc:824] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:26.101573 4195 tablet_bootstrap.cc:654] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:26.102734 4196 tablet_bootstrap.cc:654] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:26.102839 4195 log.cc:824] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:26.103924 4196 log.cc:824] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:26.104915 4194 tablet_bootstrap.cc:492] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: No bootstrap required, opened a new log
I20260618 22:37:26.105172 4194 ts_tablet_manager.cc:1397] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260618 22:37:26.105526 4195 tablet_bootstrap.cc:492] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: No bootstrap required, opened a new log
I20260618 22:37:26.105752 4195 ts_tablet_manager.cc:1397] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260618 22:37:26.106750 4196 tablet_bootstrap.cc:492] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: No bootstrap required, opened a new log
I20260618 22:37:26.106990 4196 ts_tablet_manager.cc:1397] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260618 22:37:26.113389 4195 raft_consensus.cc:348] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79f479389f4d4c8eac640c66fede48" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35829 } } peers { permanent_uuid: "0e0b2516e64c4b619e0ad44af776cd45" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45743 } } peers { permanent_uuid: "f3d963409b0f48a4be60dc261fcd1752" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 44789 } }
I20260618 22:37:26.113713 4195 raft_consensus.cc:374] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:26.113818 4195 raft_consensus.cc:729] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d963409b0f48a4be60dc261fcd1752, State: Initialized, Role: FOLLOWER
I20260618 22:37:26.114382 4195 consensus_queue.cc:260] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79f479389f4d4c8eac640c66fede48" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35829 } } peers { permanent_uuid: "0e0b2516e64c4b619e0ad44af776cd45" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45743 } } peers { permanent_uuid: "f3d963409b0f48a4be60dc261fcd1752" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 44789 } }
I20260618 22:37:26.115370 4043 heartbeater.cc:503] Master 127.18.150.190:40895 was elected leader, sending a full tablet report...
I20260618 22:37:26.115476 4194 raft_consensus.cc:348] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79f479389f4d4c8eac640c66fede48" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35829 } } peers { permanent_uuid: "0e0b2516e64c4b619e0ad44af776cd45" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45743 } } peers { permanent_uuid: "f3d963409b0f48a4be60dc261fcd1752" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 44789 } }
I20260618 22:37:26.115878 4194 raft_consensus.cc:374] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:26.115960 4194 raft_consensus.cc:729] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0b2516e64c4b619e0ad44af776cd45, State: Initialized, Role: FOLLOWER
I20260618 22:37:26.116003 4195 ts_tablet_manager.cc:1428] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260618 22:37:26.116494 4194 consensus_queue.cc:260] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79f479389f4d4c8eac640c66fede48" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35829 } } peers { permanent_uuid: "0e0b2516e64c4b619e0ad44af776cd45" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45743 } } peers { permanent_uuid: "f3d963409b0f48a4be60dc261fcd1752" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 44789 } }
I20260618 22:37:26.116914 4196 raft_consensus.cc:348] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79f479389f4d4c8eac640c66fede48" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35829 } } peers { permanent_uuid: "0e0b2516e64c4b619e0ad44af776cd45" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45743 } } peers { permanent_uuid: "f3d963409b0f48a4be60dc261fcd1752" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 44789 } }
I20260618 22:37:26.117367 4196 raft_consensus.cc:374] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:26.117492 4196 raft_consensus.cc:729] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f79f479389f4d4c8eac640c66fede48, State: Initialized, Role: FOLLOWER
I20260618 22:37:26.117522 4176 heartbeater.cc:503] Master 127.18.150.190:40895 was elected leader, sending a full tablet report...
I20260618 22:37:26.118176 4194 ts_tablet_manager.cc:1428] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260618 22:37:26.118146 4196 consensus_queue.cc:260] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79f479389f4d4c8eac640c66fede48" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35829 } } peers { permanent_uuid: "0e0b2516e64c4b619e0ad44af776cd45" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45743 } } peers { permanent_uuid: "f3d963409b0f48a4be60dc261fcd1752" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 44789 } }
I20260618 22:37:26.119685 4196 ts_tablet_manager.cc:1428] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260618 22:37:26.240800 4177 tablet.cc:2369] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:37:26.259341 3910 tablet.cc:2369] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:37:26.265419 4202 raft_consensus.cc:482] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:37:26.265699 4202 raft_consensus.cc:504] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79f479389f4d4c8eac640c66fede48" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35829 } } peers { permanent_uuid: "0e0b2516e64c4b619e0ad44af776cd45" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45743 } } peers { permanent_uuid: "f3d963409b0f48a4be60dc261fcd1752" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 44789 } }
I20260618 22:37:26.267371 4202 leader_election.cc:288] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e0b2516e64c4b619e0ad44af776cd45 (127.18.150.131:45743), f3d963409b0f48a4be60dc261fcd1752 (127.18.150.130:44789)
W20260618 22:37:26.267539 4044 tablet.cc:2369] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:37:26.273890 4192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.268233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:48095 (local address 127.18.150.131:45743)
0618 22:37:26.268417 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:26.268423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:26.268626 (+ 203us) server_negotiation.cc:407] Connection header received
0618 22:37:26.268857 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:26.268861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:26.268920 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:26.268991 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:26.269518 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.270310 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.272215 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.272955 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.273116 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.273201 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:26.273514 (+ 313us) server_negotiation.cc:299] Negotiation successful
0618 22:37:26.273644 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:37:26.273917 4193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.267942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:48815 (local address 127.18.150.130:44789)
0618 22:37:26.268328 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:26.268334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:26.269051 (+ 717us) server_negotiation.cc:407] Connection header received
0618 22:37:26.269244 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:26.269248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:26.269303 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:26.269375 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:26.269892 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.270596 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.272258 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.273008 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.273141 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.273221 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:26.273563 (+ 342us) server_negotiation.cc:299] Negotiation successful
0618 22:37:26.273682 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260618 22:37:26.274124 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.267875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:44789 (local address 127.18.150.129:48815)
0618 22:37:26.268946 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:26.268966 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:26.269098 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:26.269407 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:26.269411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:26.269434 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:26.269752 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:26.269760 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.270734 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.270738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:26.272108 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:26.272119 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.273217 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.273223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:26.273418 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.273442 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:37:26.273517 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:37:26.273642 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":31,"thread_start_us":805,"threads_started":1}
I20260618 22:37:26.274293 4203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.267848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:45743 (local address 127.18.150.129:48095)
0618 22:37:26.268514 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:26.268532 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:26.268699 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:26.269030 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:26.269034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:26.269115 (+ 81us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:26.269379 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:26.269388 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.270435 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.270439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:26.272021 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:26.272037 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.273172 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.273179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:26.273345 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.273386 (+ 41us) client_negotiation.cc:769] Sending connection context
0618 22:37:26.273463 (+ 77us) client_negotiation.cc:240] Negotiation successful
0618 22:37:26.273585 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":78,"spinlock_wait_cycles":38656,"thread_start_us":567,"threads_started":1}
I20260618 22:37:26.275094 3998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43849cde5dc462eb6d299194501ce93" candidate_uuid: "1f79f479389f4d4c8eac640c66fede48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d963409b0f48a4be60dc261fcd1752" is_pre_election: true
I20260618 22:37:26.275148 4131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43849cde5dc462eb6d299194501ce93" candidate_uuid: "1f79f479389f4d4c8eac640c66fede48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0b2516e64c4b619e0ad44af776cd45" is_pre_election: true
I20260618 22:37:26.275553 3998 raft_consensus.cc:2454] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f79f479389f4d4c8eac640c66fede48 in term 0.
I20260618 22:37:26.275560 4131 raft_consensus.cc:2454] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f79f479389f4d4c8eac640c66fede48 in term 0.
I20260618 22:37:26.276134 3800 leader_election.cc:302] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e0b2516e64c4b619e0ad44af776cd45, 1f79f479389f4d4c8eac640c66fede48; no voters:
I20260618 22:37:26.276494 4202 raft_consensus.cc:2781] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:37:26.276648 4202 raft_consensus.cc:482] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:37:26.276746 4202 raft_consensus.cc:3037] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:26.278856 4202 raft_consensus.cc:504] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79f479389f4d4c8eac640c66fede48" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35829 } } peers { permanent_uuid: "0e0b2516e64c4b619e0ad44af776cd45" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45743 } } peers { permanent_uuid: "f3d963409b0f48a4be60dc261fcd1752" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 44789 } }
I20260618 22:37:26.279470 4202 leader_election.cc:288] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [CANDIDATE]: Term 1 election: Requested vote from peers 0e0b2516e64c4b619e0ad44af776cd45 (127.18.150.131:45743), f3d963409b0f48a4be60dc261fcd1752 (127.18.150.130:44789)
I20260618 22:37:26.279856 4131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43849cde5dc462eb6d299194501ce93" candidate_uuid: "1f79f479389f4d4c8eac640c66fede48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0b2516e64c4b619e0ad44af776cd45"
I20260618 22:37:26.279965 3998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43849cde5dc462eb6d299194501ce93" candidate_uuid: "1f79f479389f4d4c8eac640c66fede48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d963409b0f48a4be60dc261fcd1752"
I20260618 22:37:26.280072 4131 raft_consensus.cc:3037] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:26.280233 3998 raft_consensus.cc:3037] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:26.283180 3998 raft_consensus.cc:2454] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f79f479389f4d4c8eac640c66fede48 in term 1.
I20260618 22:37:26.283212 4131 raft_consensus.cc:2454] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f79f479389f4d4c8eac640c66fede48 in term 1.
I20260618 22:37:26.283615 3799 leader_election.cc:302] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f79f479389f4d4c8eac640c66fede48, f3d963409b0f48a4be60dc261fcd1752; no voters:
I20260618 22:37:26.283914 4202 raft_consensus.cc:2781] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:26.284338 4202 raft_consensus.cc:686] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 1 LEADER]: Becoming Leader. State: Replica: 1f79f479389f4d4c8eac640c66fede48, State: Running, Role: LEADER
I20260618 22:37:26.284852 4202 consensus_queue.cc:237] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79f479389f4d4c8eac640c66fede48" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35829 } } peers { permanent_uuid: "0e0b2516e64c4b619e0ad44af776cd45" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45743 } } peers { permanent_uuid: "f3d963409b0f48a4be60dc261fcd1752" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 44789 } }
I20260618 22:37:26.289203 3717 catalog_manager.cc:5495] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f79f479389f4d4c8eac640c66fede48 (127.18.150.129). New cstate: current_term: 1 leader_uuid: "1f79f479389f4d4c8eac640c66fede48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f79f479389f4d4c8eac640c66fede48" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e0b2516e64c4b619e0ad44af776cd45" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d963409b0f48a4be60dc261fcd1752" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 44789 } health_report { overall_health: UNKNOWN } } }
I20260618 22:37:26.319489 4190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.314034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57810 (local address 127.18.150.129:35829)
0618 22:37:26.314230 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:26.314234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:26.314269 (+ 35us) server_negotiation.cc:407] Connection header received
0618 22:37:26.314384 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:26.314388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:26.314473 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:26.314593 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:37:26.314918 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.315613 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.317245 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.317669 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.317769 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.318368 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:37:26.318880 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:37:26.318985 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:26.319110 (+ 125us) server_negotiation.cc:299] Negotiation successful
0618 22:37:26.319185 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:37:26.327780 4131 raft_consensus.cc:1261] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45 [term 1 FOLLOWER]: Refusing update from remote peer 1f79f479389f4d4c8eac640c66fede48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:26.327776 3998 raft_consensus.cc:1261] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752 [term 1 FOLLOWER]: Refusing update from remote peer 1f79f479389f4d4c8eac640c66fede48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:26.328764 4205 consensus_queue.cc:1035] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d963409b0f48a4be60dc261fcd1752" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 44789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:37:26.329080 4202 consensus_queue.cc:1035] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0b2516e64c4b619e0ad44af776cd45" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 45743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:37:26.346081 4209 mvcc.cc:205] Tried to move back new op lower bound from 7298343920947908608 to 7298343920786792448. Current Snapshot: MvccSnapshot[applied={T|T < 7298343920947908608}]
I20260618 22:37:26.346336 4210 mvcc.cc:205] Tried to move back new op lower bound from 7298343920947908608 to 7298343920786792448. Current Snapshot: MvccSnapshot[applied={T|T < 7298343920947908608 or (T in {7298343920947908608})}]
I20260618 22:37:26.353157 3717 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:26.353405 3717 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:26.356767 3717 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f206b86ff29048f38d60fa5d22d9b0e3: Sending DeleteTablet for 3 replicas of tablet d43849cde5dc462eb6d299194501ce93
I20260618 22:37:26.357784 3844 tablet_service.cc:1514] Processing DeleteTablet for tablet d43849cde5dc462eb6d299194501ce93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:26 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:57796
I20260618 22:37:26.357817 4111 tablet_service.cc:1514] Processing DeleteTablet for tablet d43849cde5dc462eb6d299194501ce93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:26 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:45908
I20260618 22:37:26.358115 3978 tablet_service.cc:1514] Processing DeleteTablet for tablet d43849cde5dc462eb6d299194501ce93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:26 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:53734
I20260618 22:37:26.360222 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 3781
I20260618 22:37:26.366416 4217 tablet_replica.cc:330] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: stopping tablet replica
I20260618 22:37:26.366987 4217 raft_consensus.cc:2229] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:37:26.367792 4217 raft_consensus.cc:2258] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:26.373977 4218 tablet_replica.cc:330] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: stopping tablet replica
I20260618 22:37:26.374203 4219 tablet_replica.cc:330] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: stopping tablet replica
I20260618 22:37:26.374612 4219 raft_consensus.cc:2229] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:37:26.374612 4218 raft_consensus.cc:2229] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:37:26.375121 4218 raft_consensus.cc:2258] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:26.375121 4219 raft_consensus.cc:2258] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:26.376499 4217 ts_tablet_manager.cc:1905] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:26.377091 4218 ts_tablet_manager.cc:1905] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:26.382406 4219 ts_tablet_manager.cc:1905] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:26.384114 4218 ts_tablet_manager.cc:1918] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:26.384325 4218 log.cc:1196] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/d43849cde5dc462eb6d299194501ce93
I20260618 22:37:26.384866 4218 ts_tablet_manager.cc:1939] T d43849cde5dc462eb6d299194501ce93 P 0e0b2516e64c4b619e0ad44af776cd45: Deleting consensus metadata
I20260618 22:37:26.386919 3707 catalog_manager.cc:4841] TS 0e0b2516e64c4b619e0ad44af776cd45 (127.18.150.131:45743): tablet d43849cde5dc462eb6d299194501ce93 (table test-table [id=ff938b7526eb430494c1a9145fcf87ae]) successfully deleted
Jun 18 22:37:26 dist-test-slave-h7jz krb5kdc[3675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:37:26.391175 4217 ts_tablet_manager.cc:1918] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:26.391503 4217 log.cc:1196] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/d43849cde5dc462eb6d299194501ce93
I20260618 22:37:26.392503 4217 ts_tablet_manager.cc:1939] T d43849cde5dc462eb6d299194501ce93 P 1f79f479389f4d4c8eac640c66fede48: Deleting consensus metadata
I20260618 22:37:26.395191 4219 ts_tablet_manager.cc:1918] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:26.395427 4219 log.cc:1196] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/d43849cde5dc462eb6d299194501ce93
I20260618 22:37:26.395609 3704 catalog_manager.cc:4841] TS 1f79f479389f4d4c8eac640c66fede48 (127.18.150.129:35829): tablet d43849cde5dc462eb6d299194501ce93 (table test-table [id=ff938b7526eb430494c1a9145fcf87ae]) successfully deleted
I20260618 22:37:26.396006 4219 ts_tablet_manager.cc:1939] T d43849cde5dc462eb6d299194501ce93 P f3d963409b0f48a4be60dc261fcd1752: Deleting consensus metadata
I20260618 22:37:26.397795 3706 catalog_manager.cc:4841] TS f3d963409b0f48a4be60dc261fcd1752 (127.18.150.130:44789): tablet d43849cde5dc462eb6d299194501ce93 (table test-table [id=ff938b7526eb430494c1a9145fcf87ae]) successfully deleted
I20260618 22:37:26.399619 4190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.361605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57818 (local address 127.18.150.129:35829)
0618 22:37:26.361796 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:26.361801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:26.373162 (+ 11361us) server_negotiation.cc:407] Connection header received
0618 22:37:26.374036 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:26.374041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:26.374163 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:26.374242 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:26.376113 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.377170 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.379055 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.379751 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.379847 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.391262 (+ 11415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:26.391281 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:26.391301 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:26.391360 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:26.395331 (+ 3971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:26.396728 (+ 1397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:26.396738 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:26.396742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:26.396911 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:26.397346 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:26.397352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:26.397356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:26.398226 (+ 870us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:26.398515 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:26.398855 (+ 340us) server_negotiation.cc:299] Negotiation successful
0618 22:37:26.399092 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260618 22:37:26.400362 3904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:26.499874 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 3781
I20260618 22:37:26.515286 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 3915
Jun 18 22:37:26 dist-test-slave-h7jz krb5kdc[3675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:37:26.532485 4193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.515997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53736 (local address 127.18.150.130:44789)
0618 22:37:26.516164 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:26.516169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:26.516232 (+ 63us) server_negotiation.cc:407] Connection header received
0618 22:37:26.516408 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:26.516411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:26.516481 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:26.516558 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:26.518184 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.519023 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.520799 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.521203 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.521294 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.526291 (+ 4997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:26.526303 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:26.526318 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:26.526363 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:26.529351 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:26.530084 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:26.530092 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:26.530094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:26.530230 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:26.530661 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:26.530667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:26.530671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:26.531254 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:26.531597 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:26.531894 (+ 297us) server_negotiation.cc:299] Negotiation successful
0618 22:37:26.532124 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:37:26.533128 4038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:26.623201 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 3915
I20260618 22:37:26.636611 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 4048
Jun 18 22:37:26 dist-test-slave-h7jz krb5kdc[3675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:37:26.651234 4192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:26.637329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45914 (local address 127.18.150.131:45743)
0618 22:37:26.637500 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:26.637506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:26.637525 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:37:26.637705 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:26.637709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:26.637782 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:26.637879 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:26.639140 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.639970 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.641268 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:26.641859 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:26.641995 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:26.646080 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:26.646099 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:26.646115 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:26.646146 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:26.648718 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:26.649385 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:26.649393 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:26.649395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:26.649574 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:26.649894 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:26.649897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:26.649899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:26.650321 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:26.650570 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:26.650798 (+ 228us) server_negotiation.cc:299] Negotiation successful
0618 22:37:26.650964 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:37:26.651875 4171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:26.738641 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 4048
I20260618 22:37:26.751560 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 3688
I20260618 22:37:26.752693 3750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:26.861069 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 3688
2026-06-18T22:37:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4134 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-18T22:37:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:26Z Disabled control of system clock
I20260618 22:37:26.928319 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35321
--webserver_interface=127.18.150.190
--webserver_port=0
--builtin_ntp_servers=127.18.150.148:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:35321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:37:27.259572 4228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:27.259881 4228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:27.260074 4228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:27.267692 4228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:27.267805 4228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:27.267871 4228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:27.267963 4228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:27.268024 4228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:27.277467 4228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:35321
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:27.278800 4228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:27.280601 4228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:27.291957 4236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:27.292057 4234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:27.292083 4233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:27.292946 4228 server_base.cc:1034] running on GCE node
I20260618 22:37:27.293649 4228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:27.295045 4228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:27.296262 4228 hybrid_clock.cc:648] HybridClock initialized: now 1781822247296213 us; error 44 us; skew 500 ppm
I20260618 22:37:27.296604 4228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:27.298911 4228 webserver.cc:460] Webserver started at http://127.18.150.190:39665/ using document root <none> and password file <none>
I20260618 22:37:27.299564 4228 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:27.299662 4228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:27.299952 4228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:27.302765 4228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "d0216349e7f04ddea930298a2fcdf3bf"
format_stamp: "Formatted at 2026-06-18 22:37:27 on dist-test-slave-h7jz"
server_key: "045841fdf95e545b319ab6ff50a1ccf2"
server_key_iv: "8ec4788d2099c0a7ee10bb0e448f3675"
server_key_version: "encryptionkey@0"
I20260618 22:37:27.303498 4228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "d0216349e7f04ddea930298a2fcdf3bf"
format_stamp: "Formatted at 2026-06-18 22:37:27 on dist-test-slave-h7jz"
server_key: "045841fdf95e545b319ab6ff50a1ccf2"
server_key_iv: "8ec4788d2099c0a7ee10bb0e448f3675"
server_key_version: "encryptionkey@0"
I20260618 22:37:27.308259 4228 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:37:27.311612 4242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:27.312928 4228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:37:27.313115 4228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "d0216349e7f04ddea930298a2fcdf3bf"
format_stamp: "Formatted at 2026-06-18 22:37:27 on dist-test-slave-h7jz"
server_key: "045841fdf95e545b319ab6ff50a1ccf2"
server_key_iv: "8ec4788d2099c0a7ee10bb0e448f3675"
server_key_version: "encryptionkey@0"
I20260618 22:37:27.313319 4228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:27.330169 4228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:27.330976 4228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:27.331260 4228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:27.349406 4228 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:35321
I20260618 22:37:27.349424 4293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:35321 every 8 connection(s)
I20260618 22:37:27.350822 4228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:27.354518 4294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:27.358582 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 4228
I20260618 22:37:27.358763 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:27.359295 19034 external_mini_cluster.cc:1428] Setting key 2e726bd7d3747e711bb09cd57a8be6d8
I20260618 22:37:27.365626 4294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf: Bootstrap starting.
I20260618 22:37:27.369045 4294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:27.370241 4294 log.cc:824] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:27.373171 4294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf: No bootstrap required, opened a new log
I20260618 22:37:27.373309 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:27.361143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.18.150.190:35321)
0618 22:37:27.362106 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:27.362119 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:27.362164 (+ 45us) server_negotiation.cc:407] Connection header received
0618 22:37:27.364111 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:27.364135 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:27.364568 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:27.364916 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:27.365481 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:27.366828 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:27.368805 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:27.369531 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:27.369679 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:27.370886 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:27.370899 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:27.370916 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:27.371608 (+ 692us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:27.371696 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:27.371717 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:27.371798 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:27.372062 (+ 264us) server_negotiation.cc:299] Negotiation successful
0618 22:37:27.372380 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":332,"threads_started":1}
I20260618 22:37:27.380934 4294 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0216349e7f04ddea930298a2fcdf3bf" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35321 } }
I20260618 22:37:27.381305 4294 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:27.381397 4294 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0216349e7f04ddea930298a2fcdf3bf, State: Initialized, Role: FOLLOWER
I20260618 22:37:27.381966 4294 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0216349e7f04ddea930298a2fcdf3bf" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35321 } }
I20260618 22:37:27.382193 4294 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:27.382323 4294 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:27.382472 4294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:27.384853 4294 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0216349e7f04ddea930298a2fcdf3bf" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35321 } }
I20260618 22:37:27.385391 4294 leader_election.cc:302] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0216349e7f04ddea930298a2fcdf3bf; no voters:
I20260618 22:37:27.385987 4294 leader_election.cc:288] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:27.386142 4299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:27.386819 4299 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [term 1 LEADER]: Becoming Leader. State: Replica: d0216349e7f04ddea930298a2fcdf3bf, State: Running, Role: LEADER
I20260618 22:37:27.387420 4299 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0216349e7f04ddea930298a2fcdf3bf" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35321 } }
I20260618 22:37:27.388062 4294 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:27.390278 4300 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0216349e7f04ddea930298a2fcdf3bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0216349e7f04ddea930298a2fcdf3bf" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35321 } } }
I20260618 22:37:27.390410 4301 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0216349e7f04ddea930298a2fcdf3bf. Latest consensus state: current_term: 1 leader_uuid: "d0216349e7f04ddea930298a2fcdf3bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0216349e7f04ddea930298a2fcdf3bf" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35321 } } }
I20260618 22:37:27.390666 4301 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:27.390671 4300 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:27.392278 4306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:27.397603 4306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:27.412573 4306 catalog_manager.cc:1347] Generated new cluster ID: 195a22a194774da386490de0fa9e3632
I20260618 22:37:27.412693 4306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:27.432693 4306 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:27.434506 4306 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:27.444839 4306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf: Generated new TSK 0
I20260618 22:37:27.445639 4306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:27.456946 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35321
--builtin_ntp_servers=127.18.150.148:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:37:27.774601 4318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:27.774899 4318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:27.775022 4318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:27.782109 4318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:27.782348 4318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:27.782539 4318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:37:27.791819 4318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:27.793118 4318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:27.794950 4318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:27.807534 4324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:27.807574 4323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:27.807600 4326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:27.808578 4318 server_base.cc:1034] running on GCE node
I20260618 22:37:27.809186 4318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:27.810174 4318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:27.811398 4318 hybrid_clock.cc:648] HybridClock initialized: now 1781822247811350 us; error 39 us; skew 500 ppm
I20260618 22:37:27.811779 4318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:27.819490 4318 webserver.cc:460] Webserver started at http://127.18.150.129:44417/ using document root <none> and password file <none>
I20260618 22:37:27.820277 4318 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:27.820389 4318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:27.820695 4318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:27.823782 4318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "a48d3456b54f432d8dbeb78e993facfa"
format_stamp: "Formatted at 2026-06-18 22:37:27 on dist-test-slave-h7jz"
server_key: "d1db61b58df883da6dece27f174054d6"
server_key_iv: "b31ed681428f39d9c3a7b61909b4842b"
server_key_version: "encryptionkey@0"
I20260618 22:37:27.824615 4318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "a48d3456b54f432d8dbeb78e993facfa"
format_stamp: "Formatted at 2026-06-18 22:37:27 on dist-test-slave-h7jz"
server_key: "d1db61b58df883da6dece27f174054d6"
server_key_iv: "b31ed681428f39d9c3a7b61909b4842b"
server_key_version: "encryptionkey@0"
I20260618 22:37:27.830057 4318 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260618 22:37:27.833698 4332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:27.835172 4318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:37:27.835352 4318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "a48d3456b54f432d8dbeb78e993facfa"
format_stamp: "Formatted at 2026-06-18 22:37:27 on dist-test-slave-h7jz"
server_key: "d1db61b58df883da6dece27f174054d6"
server_key_iv: "b31ed681428f39d9c3a7b61909b4842b"
server_key_version: "encryptionkey@0"
I20260618 22:37:27.835556 4318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:27.856597 4318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:27.857446 4318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:27.857735 4318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:27.858749 4318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:27.860255 4318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:27.860349 4318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:27.860453 4318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:27.860520 4318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:27.890353 4318 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:45839
I20260618 22:37:27.890407 4444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:45839 every 8 connection(s)
I20260618 22:37:27.891595 4318 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:37:27.898777 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 4318
I20260618 22:37:27.898939 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:37:27.899360 19034 external_mini_cluster.cc:1428] Setting key fbf14b9fa7d2a9f047c6c8553d6a7efc
I20260618 22:37:27.902009 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35321
--builtin_ntp_servers=127.18.150.148:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:37:27.904031 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:27.894086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:34751 (local address 127.18.150.190:35321)
0618 22:37:27.894522 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:27.894528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:27.895792 (+ 1264us) server_negotiation.cc:407] Connection header received
0618 22:37:27.897523 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:27.897527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:27.897592 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:27.897672 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:27.898745 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:27.899549 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:27.901352 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:27.901765 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:27.901874 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:27.903237 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:27.903244 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:27.903249 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:27.903265 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:27.903307 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:27.903318 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:27.903393 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:27.903612 (+ 219us) server_negotiation.cc:299] Negotiation successful
0618 22:37:27.903713 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":226,"threads_started":1}
I20260618 22:37:27.905028 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:27.894722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35321 (local address 127.18.150.129:34751)
0618 22:37:27.895636 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:27.895681 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:27.897272 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:27.897863 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:27.897871 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:27.897989 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:27.898564 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:27.898581 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:27.899717 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:27.899721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:27.901213 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:27.901232 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:27.902010 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:27.902018 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:27.902292 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:27.902931 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:37:27.902933 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:27.902972 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:37:27.903058 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:27.903406 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:27.903417 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:27.903425 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:37:27.903570 (+ 145us) client_negotiation.cc:240] Negotiation successful
0618 22:37:27.903982 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":260,"threads_started":1}
I20260618 22:37:27.906564 4445 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35321
I20260618 22:37:27.906858 4445 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:27.907567 4445 heartbeater.cc:511] Master 127.18.150.190:35321 requested a full tablet report, sending...
I20260618 22:37:27.909551 4259 ts_manager.cc:194] Registered new tserver with Master: a48d3456b54f432d8dbeb78e993facfa (127.18.150.129:45839)
I20260618 22:37:27.911051 4259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.129:34751
W20260618 22:37:28.214152 4449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:28.214448 4449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:28.214576 4449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:28.221832 4449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:28.221961 4449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:28.222093 4449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:37:28.230962 4449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:28.232189 4449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:28.233924 4449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:28.246325 4454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:28.246371 4457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:28.246297 4455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:28.247164 4449 server_base.cc:1034] running on GCE node
I20260618 22:37:28.247747 4449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:28.248668 4449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:28.249872 4449 hybrid_clock.cc:648] HybridClock initialized: now 1781822248249826 us; error 36 us; skew 500 ppm
I20260618 22:37:28.250170 4449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:28.252272 4449 webserver.cc:460] Webserver started at http://127.18.150.130:36699/ using document root <none> and password file <none>
I20260618 22:37:28.252892 4449 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:28.252997 4449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:28.253300 4449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:28.256003 4449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda"
format_stamp: "Formatted at 2026-06-18 22:37:28 on dist-test-slave-h7jz"
server_key: "967e6f2dd7245c3c7a3deb31de3f8da9"
server_key_iv: "66b04678940fb0646edc0782b4669ead"
server_key_version: "encryptionkey@0"
I20260618 22:37:28.256724 4449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda"
format_stamp: "Formatted at 2026-06-18 22:37:28 on dist-test-slave-h7jz"
server_key: "967e6f2dd7245c3c7a3deb31de3f8da9"
server_key_iv: "66b04678940fb0646edc0782b4669ead"
server_key_version: "encryptionkey@0"
I20260618 22:37:28.261409 4449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260618 22:37:28.264588 4463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:28.265816 4449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:37:28.265981 4449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda"
format_stamp: "Formatted at 2026-06-18 22:37:28 on dist-test-slave-h7jz"
server_key: "967e6f2dd7245c3c7a3deb31de3f8da9"
server_key_iv: "66b04678940fb0646edc0782b4669ead"
server_key_version: "encryptionkey@0"
I20260618 22:37:28.266168 4449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:28.277765 4449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:28.278528 4449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:28.278786 4449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:28.279631 4449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:28.281060 4449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:28.281149 4449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:28.281261 4449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:28.281330 4449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:28.309721 4449 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:34371
I20260618 22:37:28.309772 4575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:34371 every 8 connection(s)
I20260618 22:37:28.310983 4449 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:37:28.314368 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 4449
I20260618 22:37:28.314550 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:37:28.315115 19034 external_mini_cluster.cc:1428] Setting key bc544507fd0e76165017c11bf415a783
I20260618 22:37:28.317849 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35321
--builtin_ntp_servers=127.18.150.148:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:37:28.323498 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.313681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:37435 (local address 127.18.150.190:35321)
0618 22:37:28.314022 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:28.314031 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:28.315463 (+ 1432us) server_negotiation.cc:407] Connection header received
0618 22:37:28.317454 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:28.317460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:28.317541 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:28.317632 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:28.318759 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.319461 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.320957 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.321357 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.321474 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.322731 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:28.322737 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:28.322742 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:28.322760 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:28.322792 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:28.322803 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:28.322874 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:28.323095 (+ 221us) server_negotiation.cc:299] Negotiation successful
0618 22:37:28.323193 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260618 22:37:28.324471 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.314322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35321 (local address 127.18.150.130:37435)
0618 22:37:28.315314 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:28.315352 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:28.317212 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:28.317882 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:28.317894 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:28.318039 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:28.318603 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.318622 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.319612 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.319616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.320825 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.320837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.321567 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.321575 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.321787 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.322428 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:37:28.322431 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:28.322474 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:37:28.322551 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:28.322886 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:28.322897 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:28.322905 (+ 8us) client_negotiation.cc:769] Sending connection context
0618 22:37:28.323055 (+ 150us) client_negotiation.cc:240] Negotiation successful
0618 22:37:28.323439 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":306,"threads_started":1}
I20260618 22:37:28.325868 4576 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35321
I20260618 22:37:28.326193 4576 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:28.327029 4576 heartbeater.cc:511] Master 127.18.150.190:35321 requested a full tablet report, sending...
I20260618 22:37:28.328471 4259 ts_manager.cc:194] Registered new tserver with Master: a3eea1fdd41b463d8f2c1f7f07aebcda (127.18.150.130:34371)
I20260618 22:37:28.329278 4259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.130:37435
W20260618 22:37:28.631194 4579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:28.631457 4579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:28.631551 4579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:28.638630 4579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:28.638722 4579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:28.638840 4579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:37:28.647835 4579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:35321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:28.649008 4579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:28.650673 4579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:28.662662 4584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:28.662621 4585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:28.662688 4587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:28.663470 4579 server_base.cc:1034] running on GCE node
I20260618 22:37:28.663991 4579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:28.664817 4579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:28.666040 4579 hybrid_clock.cc:648] HybridClock initialized: now 1781822248665980 us; error 45 us; skew 500 ppm
I20260618 22:37:28.666328 4579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:28.668361 4579 webserver.cc:460] Webserver started at http://127.18.150.131:38097/ using document root <none> and password file <none>
I20260618 22:37:28.669039 4579 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:28.669123 4579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:28.669363 4579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:28.672191 4579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "9a45b48787cc4363bb39deee0146be88"
format_stamp: "Formatted at 2026-06-18 22:37:28 on dist-test-slave-h7jz"
server_key: "28266ceae30818e0679c23d06c67ef2a"
server_key_iv: "2eaf114442733a1a9c0c1a24d133300e"
server_key_version: "encryptionkey@0"
I20260618 22:37:28.672914 4579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "9a45b48787cc4363bb39deee0146be88"
format_stamp: "Formatted at 2026-06-18 22:37:28 on dist-test-slave-h7jz"
server_key: "28266ceae30818e0679c23d06c67ef2a"
server_key_iv: "2eaf114442733a1a9c0c1a24d133300e"
server_key_version: "encryptionkey@0"
I20260618 22:37:28.677641 4579 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260618 22:37:28.680972 4593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:28.682458 4579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260618 22:37:28.682588 4579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "9a45b48787cc4363bb39deee0146be88"
format_stamp: "Formatted at 2026-06-18 22:37:28 on dist-test-slave-h7jz"
server_key: "28266ceae30818e0679c23d06c67ef2a"
server_key_iv: "2eaf114442733a1a9c0c1a24d133300e"
server_key_version: "encryptionkey@0"
I20260618 22:37:28.682772 4579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:28.704052 4579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:28.704859 4579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:28.705116 4579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:28.706007 4579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:28.707405 4579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:28.707487 4579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:28.707563 4579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:28.707613 4579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:28.736532 4579 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:33987
I20260618 22:37:28.736603 4705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:33987 every 8 connection(s)
I20260618 22:37:28.737780 4579 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:37:28.741043 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 4579
I20260618 22:37:28.741226 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:37:28.741840 19034 external_mini_cluster.cc:1428] Setting key 020c46c0c92232ca4db609fa464dc500
I20260618 22:37:28.750129 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.740241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:46711 (local address 127.18.150.190:35321)
0618 22:37:28.740417 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:28.740421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:28.742081 (+ 1660us) server_negotiation.cc:407] Connection header received
0618 22:37:28.744181 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:28.744186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:28.744268 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:28.744372 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:28.745459 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.746237 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.747695 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.748087 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.748173 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.749334 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:28.749339 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:28.749342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:28.749359 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:28.749390 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:28.749401 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:28.749464 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:28.749630 (+ 166us) server_negotiation.cc:299] Negotiation successful
0618 22:37:28.749795 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:37:28.750939 4708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.740825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:35321 (local address 127.18.150.131:46711)
0618 22:37:28.741885 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:28.741969 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:28.743940 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:28.744570 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:28.744581 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:28.744708 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:28.745313 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.745329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.746368 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.746372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.747585 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.747596 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.748239 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.748243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.748435 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.749050 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:37:28.749052 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:28.749090 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:37:28.749170 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:28.749472 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:28.749478 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:28.749483 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:37:28.749613 (+ 130us) client_negotiation.cc:240] Negotiation successful
0618 22:37:28.749912 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":284,"threads_started":1}
I20260618 22:37:28.752187 4706 heartbeater.cc:347] Connected to a master server at 127.18.150.190:35321
I20260618 22:37:28.752475 4706 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:28.753209 4706 heartbeater.cc:511] Master 127.18.150.190:35321 requested a full tablet report, sending...
I20260618 22:37:28.754509 4259 ts_manager.cc:194] Registered new tserver with Master: 9a45b48787cc4363bb39deee0146be88 (127.18.150.131:33987)
I20260618 22:37:28.755438 4259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.131:46711
I20260618 22:37:28.756772 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:37:28.765647 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.760484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.18.150.190:35321)
0618 22:37:28.760622 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:28.760626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:28.761023 (+ 397us) server_negotiation.cc:407] Connection header received
0618 22:37:28.761145 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:28.761148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:28.761221 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:28.761307 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:28.761761 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.762472 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.763703 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.764168 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.764268 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.765148 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:28.765152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:28.765155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:28.765171 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:28.765201 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:28.765212 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:28.765290 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:28.765369 (+ 79us) server_negotiation.cc:299] Negotiation successful
0618 22:37:28.765445 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:37:28.768836 4259 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:37:28.775851 4259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:37:28.796907 4721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.788556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59920 (local address 127.18.150.129:45839)
0618 22:37:28.789423 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:28.789432 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:28.789465 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:37:28.789566 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:28.789575 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:28.789847 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:28.790076 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:28.790716 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.792023 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.794658 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.795780 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.795930 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.796153 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:28.796408 (+ 255us) server_negotiation.cc:299] Negotiation successful
0618 22:37:28.796577 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":565,"threads_started":1}
I20260618 22:37:28.796911 4720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.788308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:45839 (local address 127.0.0.1:59920)
0618 22:37:28.788833 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:28.788856 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:28.788963 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:28.790080 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:28.790088 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:28.790117 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:28.790528 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.790540 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.792166 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.792171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.794433 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.794452 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.795969 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.795982 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.796200 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.796246 (+ 46us) client_negotiation.cc:769] Sending connection context
0618 22:37:28.796360 (+ 114us) client_negotiation.cc:240] Negotiation successful
0618 22:37:28.796541 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":21,"thread_start_us":247,"threads_started":1}
I20260618 22:37:28.798503 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.789329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:33987 (local address 127.0.0.1:44372)
0618 22:37:28.789741 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:28.789762 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:28.789864 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:28.791578 (+ 1714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:28.791583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:28.791611 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:28.791986 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.791997 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.793779 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.793784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.795999 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.796013 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.797754 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.797760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.797921 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.797976 (+ 55us) client_negotiation.cc:769] Sending connection context
0618 22:37:28.798065 (+ 89us) client_negotiation.cc:240] Negotiation successful
0618 22:37:28.798185 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":20,"thread_start_us":824,"threads_started":1}
I20260618 22:37:28.798623 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.789863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44372 (local address 127.18.150.131:33987)
0618 22:37:28.790700 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:28.790709 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:28.790742 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:37:28.790860 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:28.790868 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:28.791111 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:28.791306 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:28.792164 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.793661 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.796217 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.797590 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.797761 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.797989 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:28.798105 (+ 116us) server_negotiation.cc:299] Negotiation successful
0618 22:37:28.798276 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":515,"threads_started":1}
I20260618 22:37:28.799821 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.787199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:34371 (local address 127.0.0.1:57510)
0618 22:37:28.787993 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:28.788038 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:28.788210 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:28.792897 (+ 4687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:28.792902 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:28.792925 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:28.793350 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.793360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.795201 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.795206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.797213 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.797227 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.799081 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.799088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.799248 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.799288 (+ 40us) client_negotiation.cc:769] Sending connection context
0618 22:37:28.799368 (+ 80us) client_negotiation.cc:240] Negotiation successful
0618 22:37:28.799492 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":1,"thread_start_us":307,"threads_started":1}
I20260618 22:37:28.801234 4380 tablet_service.cc:1467] Processing CreateTablet for tablet f232d39cbabc43829ba7ee051f315bf3 (DEFAULT_TABLE table=test-table [id=0c2064d0d8854d51878320482c6515f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:28.801925 4641 tablet_service.cc:1467] Processing CreateTablet for tablet f232d39cbabc43829ba7ee051f315bf3 (DEFAULT_TABLE table=test-table [id=0c2064d0d8854d51878320482c6515f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:28.802563 4380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f232d39cbabc43829ba7ee051f315bf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:28.803299 4641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f232d39cbabc43829ba7ee051f315bf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:28.807497 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.787504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57510 (local address 127.18.150.130:34371)
0618 22:37:28.792471 (+ 4967us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:28.792477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:28.792498 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:37:28.792571 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:28.792576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:28.792721 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:28.792881 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:28.793526 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.795056 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.797464 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.798902 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.799082 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.799274 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:28.799404 (+ 130us) server_negotiation.cc:299] Negotiation successful
0618 22:37:28.799559 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4837,"thread_start_us":4730,"threads_started":1}
I20260618 22:37:28.811525 4511 tablet_service.cc:1467] Processing CreateTablet for tablet f232d39cbabc43829ba7ee051f315bf3 (DEFAULT_TABLE table=test-table [id=0c2064d0d8854d51878320482c6515f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:28.812996 4511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f232d39cbabc43829ba7ee051f315bf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:28.815459 4724 tablet_bootstrap.cc:492] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: Bootstrap starting.
I20260618 22:37:28.817267 4725 tablet_bootstrap.cc:492] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa: Bootstrap starting.
I20260618 22:37:28.819697 4724 tablet_bootstrap.cc:654] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:28.820981 4724 log.cc:824] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:28.823071 4725 tablet_bootstrap.cc:654] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:28.824147 4725 log.cc:824] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:28.826004 4724 tablet_bootstrap.cc:492] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: No bootstrap required, opened a new log
I20260618 22:37:28.826318 4728 tablet_bootstrap.cc:492] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda: Bootstrap starting.
I20260618 22:37:28.826329 4724 ts_tablet_manager.cc:1397] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260618 22:37:28.827121 4725 tablet_bootstrap.cc:492] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa: No bootstrap required, opened a new log
I20260618 22:37:28.827370 4725 ts_tablet_manager.cc:1397] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260618 22:37:28.828830 4728 tablet_bootstrap.cc:654] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:28.829753 4728 log.cc:824] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:28.833021 4728 tablet_bootstrap.cc:492] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda: No bootstrap required, opened a new log
I20260618 22:37:28.833258 4728 ts_tablet_manager.cc:1397] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260618 22:37:28.836592 4724 raft_consensus.cc:348] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 34371 } } peers { permanent_uuid: "a48d3456b54f432d8dbeb78e993facfa" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 45839 } } peers { permanent_uuid: "9a45b48787cc4363bb39deee0146be88" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33987 } }
I20260618 22:37:28.837046 4724 raft_consensus.cc:374] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:28.837188 4724 raft_consensus.cc:729] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a45b48787cc4363bb39deee0146be88, State: Initialized, Role: FOLLOWER
I20260618 22:37:28.837347 4725 raft_consensus.cc:348] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 34371 } } peers { permanent_uuid: "a48d3456b54f432d8dbeb78e993facfa" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 45839 } } peers { permanent_uuid: "9a45b48787cc4363bb39deee0146be88" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33987 } }
I20260618 22:37:28.837738 4725 raft_consensus.cc:374] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:28.837844 4725 raft_consensus.cc:729] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48d3456b54f432d8dbeb78e993facfa, State: Initialized, Role: FOLLOWER
I20260618 22:37:28.837846 4724 consensus_queue.cc:260] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 34371 } } peers { permanent_uuid: "a48d3456b54f432d8dbeb78e993facfa" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 45839 } } peers { permanent_uuid: "9a45b48787cc4363bb39deee0146be88" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33987 } }
I20260618 22:37:28.838430 4725 consensus_queue.cc:260] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 34371 } } peers { permanent_uuid: "a48d3456b54f432d8dbeb78e993facfa" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 45839 } } peers { permanent_uuid: "9a45b48787cc4363bb39deee0146be88" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33987 } }
I20260618 22:37:28.839640 4445 heartbeater.cc:503] Master 127.18.150.190:35321 was elected leader, sending a full tablet report...
I20260618 22:37:28.841053 4728 raft_consensus.cc:348] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 34371 } } peers { permanent_uuid: "a48d3456b54f432d8dbeb78e993facfa" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 45839 } } peers { permanent_uuid: "9a45b48787cc4363bb39deee0146be88" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33987 } }
I20260618 22:37:28.841527 4728 raft_consensus.cc:374] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:28.841691 4728 raft_consensus.cc:729] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3eea1fdd41b463d8f2c1f7f07aebcda, State: Initialized, Role: FOLLOWER
I20260618 22:37:28.842373 4728 consensus_queue.cc:260] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 34371 } } peers { permanent_uuid: "a48d3456b54f432d8dbeb78e993facfa" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 45839 } } peers { permanent_uuid: "9a45b48787cc4363bb39deee0146be88" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33987 } }
I20260618 22:37:28.843340 4576 heartbeater.cc:503] Master 127.18.150.190:35321 was elected leader, sending a full tablet report...
I20260618 22:37:28.844030 4728 ts_tablet_manager.cc:1428] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260618 22:37:28.845021 4706 heartbeater.cc:503] Master 127.18.150.190:35321 was elected leader, sending a full tablet report...
I20260618 22:37:28.845516 4725 ts_tablet_manager.cc:1428] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa: Time spent starting tablet: real 0.018s user 0.013s sys 0.000s
I20260618 22:37:28.845541 4724 ts_tablet_manager.cc:1428] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: Time spent starting tablet: real 0.019s user 0.013s sys 0.005s
W20260618 22:37:28.893591 4446 tablet.cc:2369] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:37:28.986079 4732 raft_consensus.cc:482] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:37:28.986385 4732 raft_consensus.cc:504] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 34371 } } peers { permanent_uuid: "a48d3456b54f432d8dbeb78e993facfa" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 45839 } } peers { permanent_uuid: "9a45b48787cc4363bb39deee0146be88" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33987 } }
I20260618 22:37:28.988080 4732 leader_election.cc:288] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a48d3456b54f432d8dbeb78e993facfa (127.18.150.129:45839), 9a45b48787cc4363bb39deee0146be88 (127.18.150.131:33987)
W20260618 22:37:28.989439 4707 tablet.cc:2369] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:37:28.995059 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.988575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:56247 (local address 127.18.150.131:33987)
0618 22:37:28.988779 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:28.988786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:28.989328 (+ 542us) server_negotiation.cc:407] Connection header received
0618 22:37:28.989630 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:28.989636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:28.989726 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:28.989788 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:28.990556 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.991319 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.993360 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.994217 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.994386 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.994452 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:28.994703 (+ 251us) server_negotiation.cc:299] Negotiation successful
0618 22:37:28.994821 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:37:28.995103 4721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.988415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:45671 (local address 127.18.150.129:45839)
0618 22:37:28.988631 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:28.988636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:28.989710 (+ 1074us) server_negotiation.cc:407] Connection header received
0618 22:37:28.989903 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:28.989908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:28.990019 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:28.990108 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:28.990744 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.991420 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.993346 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.994180 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.994303 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.994373 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:28.994727 (+ 354us) server_negotiation.cc:299] Negotiation successful
0618 22:37:28.994854 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:37:28.995267 4733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.988466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:33987 (local address 127.18.150.130:56247)
0618 22:37:28.989213 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:28.989235 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:28.989460 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:28.989996 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:28.990001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:28.990026 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:28.990418 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.990428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.991441 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.991446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.993178 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.993191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.994405 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.994412 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.994562 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.994592 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:37:28.994667 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:37:28.994865 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":119,"spinlock_wait_cycles":6400,"thread_start_us":375,"threads_started":1}
I20260618 22:37:28.995244 4734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:28.988874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:45839 (local address 127.18.150.130:45671)
0618 22:37:28.989596 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:28.989614 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:28.989768 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:28.990140 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:28.990145 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:28.990202 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:28.990582 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.990594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.991544 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.991548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.993166 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:28.993177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:28.994363 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:28.994370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:28.994525 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:28.994583 (+ 58us) client_negotiation.cc:769] Sending connection context
0618 22:37:28.994667 (+ 84us) client_negotiation.cc:240] Negotiation successful
0618 22:37:28.994853 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":105,"spinlock_wait_cycles":5376,"thread_start_us":393,"threads_started":1}
I20260618 22:37:28.996181 4661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f232d39cbabc43829ba7ee051f315bf3" candidate_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a45b48787cc4363bb39deee0146be88" is_pre_election: true
I20260618 22:37:28.996256 4400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f232d39cbabc43829ba7ee051f315bf3" candidate_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48d3456b54f432d8dbeb78e993facfa" is_pre_election: true
I20260618 22:37:28.996630 4661 raft_consensus.cc:2454] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3eea1fdd41b463d8f2c1f7f07aebcda in term 0.
I20260618 22:37:28.996707 4400 raft_consensus.cc:2454] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3eea1fdd41b463d8f2c1f7f07aebcda in term 0.
I20260618 22:37:28.997275 4464 leader_election.cc:302] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a45b48787cc4363bb39deee0146be88, a3eea1fdd41b463d8f2c1f7f07aebcda; no voters:
I20260618 22:37:28.997611 4732 raft_consensus.cc:2781] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:37:28.997730 4732 raft_consensus.cc:482] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:37:28.997821 4732 raft_consensus.cc:3037] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:28.999917 4732 raft_consensus.cc:504] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 34371 } } peers { permanent_uuid: "a48d3456b54f432d8dbeb78e993facfa" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 45839 } } peers { permanent_uuid: "9a45b48787cc4363bb39deee0146be88" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33987 } }
I20260618 22:37:29.000494 4732 leader_election.cc:288] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [CANDIDATE]: Term 1 election: Requested vote from peers a48d3456b54f432d8dbeb78e993facfa (127.18.150.129:45839), 9a45b48787cc4363bb39deee0146be88 (127.18.150.131:33987)
I20260618 22:37:29.000908 4400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f232d39cbabc43829ba7ee051f315bf3" candidate_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48d3456b54f432d8dbeb78e993facfa"
I20260618 22:37:29.001078 4400 raft_consensus.cc:3037] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:29.001134 4661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f232d39cbabc43829ba7ee051f315bf3" candidate_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a45b48787cc4363bb39deee0146be88"
I20260618 22:37:29.001343 4661 raft_consensus.cc:3037] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:29.003276 4400 raft_consensus.cc:2454] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3eea1fdd41b463d8f2c1f7f07aebcda in term 1.
I20260618 22:37:29.003628 4466 leader_election.cc:302] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3eea1fdd41b463d8f2c1f7f07aebcda, a48d3456b54f432d8dbeb78e993facfa; no voters:
I20260618 22:37:29.003635 4661 raft_consensus.cc:2454] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3eea1fdd41b463d8f2c1f7f07aebcda in term 1.
I20260618 22:37:29.003925 4732 raft_consensus.cc:2781] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:29.004391 4732 raft_consensus.cc:686] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 1 LEADER]: Becoming Leader. State: Replica: a3eea1fdd41b463d8f2c1f7f07aebcda, State: Running, Role: LEADER
I20260618 22:37:29.004983 4732 consensus_queue.cc:237] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 34371 } } peers { permanent_uuid: "a48d3456b54f432d8dbeb78e993facfa" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 45839 } } peers { permanent_uuid: "9a45b48787cc4363bb39deee0146be88" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33987 } }
I20260618 22:37:29.009410 4257 catalog_manager.cc:5495] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda reported cstate change: term changed from 0 to 1, leader changed from <none> to a3eea1fdd41b463d8f2c1f7f07aebcda (127.18.150.130). New cstate: current_term: 1 leader_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3eea1fdd41b463d8f2c1f7f07aebcda" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 34371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a48d3456b54f432d8dbeb78e993facfa" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 45839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a45b48787cc4363bb39deee0146be88" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33987 } health_report { overall_health: UNKNOWN } } }
I20260618 22:37:29.037735 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:29.032547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57526 (local address 127.18.150.130:34371)
0618 22:37:29.032749 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:29.032755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:29.032816 (+ 61us) server_negotiation.cc:407] Connection header received
0618 22:37:29.032982 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:29.032985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:29.033062 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:29.033179 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:37:29.033463 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:29.034231 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:29.035682 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:29.036129 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:29.036234 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:29.036685 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:37:29.037220 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:37:29.037347 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:29.037388 (+ 41us) server_negotiation.cc:299] Negotiation successful
0618 22:37:29.037457 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260618 22:37:29.045354 4661 raft_consensus.cc:1261] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88 [term 1 FOLLOWER]: Refusing update from remote peer a3eea1fdd41b463d8f2c1f7f07aebcda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:29.045354 4400 raft_consensus.cc:1261] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa [term 1 FOLLOWER]: Refusing update from remote peer a3eea1fdd41b463d8f2c1f7f07aebcda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:29.046237 4735 consensus_queue.cc:1035] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a45b48787cc4363bb39deee0146be88" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:37:29.046555 4732 consensus_queue.cc:1035] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [LEADER]: Connected to new peer: Peer: permanent_uuid: "a48d3456b54f432d8dbeb78e993facfa" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 45839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:37:29.055470 4738 mvcc.cc:205] Tried to move back new op lower bound from 7298343932080930816 to 7298343931928481792. Current Snapshot: MvccSnapshot[applied={T|T < 7298343932080930816}]
I20260618 22:37:29.060359 4740 mvcc.cc:205] Tried to move back new op lower bound from 7298343932080930816 to 7298343931928481792. Current Snapshot: MvccSnapshot[applied={T|T < 7298343932080930816}]
I20260618 22:37:29.063721 4739 mvcc.cc:205] Tried to move back new op lower bound from 7298343932080930816 to 7298343931928481792. Current Snapshot: MvccSnapshot[applied={T|T < 7298343932080930816}]
I20260618 22:37:29.071199 4259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:29.071458 4259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:29.074536 4259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0216349e7f04ddea930298a2fcdf3bf: Sending DeleteTablet for 3 replicas of tablet f232d39cbabc43829ba7ee051f315bf3
I20260618 22:37:29.075485 4511 tablet_service.cc:1514] Processing DeleteTablet for tablet f232d39cbabc43829ba7ee051f315bf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:29 UTC) from {username='slave'} at 127.0.0.1:57510
I20260618 22:37:29.075587 4641 tablet_service.cc:1514] Processing DeleteTablet for tablet f232d39cbabc43829ba7ee051f315bf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:29 UTC) from {username='slave'} at 127.0.0.1:44372
I20260618 22:37:29.075654 4380 tablet_service.cc:1514] Processing DeleteTablet for tablet f232d39cbabc43829ba7ee051f315bf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:29 UTC) from {username='slave'} at 127.0.0.1:59920
I20260618 22:37:29.076309 4748 tablet_replica.cc:330] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda: stopping tablet replica
I20260618 22:37:29.076565 4749 tablet_replica.cc:330] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: stopping tablet replica
I20260618 22:37:29.076704 4748 raft_consensus.cc:2229] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:37:29.077000 4749 raft_consensus.cc:2229] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:37:29.077296 4748 raft_consensus.cc:2258] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:29.077526 4749 raft_consensus.cc:2258] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:29.077669 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 4318
I20260618 22:37:29.080893 4749 ts_tablet_manager.cc:1905] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:29.082537 4748 ts_tablet_manager.cc:1905] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:29.086688 4721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:29.078740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59922 (local address 127.18.150.129:45839)
0618 22:37:29.078904 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:29.078910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:29.078929 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:37:29.079127 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:29.079132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:29.079249 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:29.079334 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:29.079988 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:29.081113 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:29.082965 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:29.083558 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:29.083648 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:29.085041 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:29.085052 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:29.085068 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:29.085782 (+ 714us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:29.085839 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:29.085862 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:29.085972 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:29.086208 (+ 236us) server_negotiation.cc:299] Negotiation successful
0618 22:37:29.086320 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:37:29.087818 4440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:29.092816 4748 ts_tablet_manager.cc:1918] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:29.092967 4748 log.cc:1196] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/f232d39cbabc43829ba7ee051f315bf3
I20260618 22:37:29.093431 4748 ts_tablet_manager.cc:1939] T f232d39cbabc43829ba7ee051f315bf3 P a3eea1fdd41b463d8f2c1f7f07aebcda: Deleting consensus metadata
I20260618 22:37:29.096846 4749 ts_tablet_manager.cc:1918] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:29.096989 4749 log.cc:1196] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/f232d39cbabc43829ba7ee051f315bf3
I20260618 22:37:29.097260 4245 catalog_manager.cc:4841] TS a3eea1fdd41b463d8f2c1f7f07aebcda (127.18.150.130:34371): tablet f232d39cbabc43829ba7ee051f315bf3 (table test-table [id=0c2064d0d8854d51878320482c6515f4]) successfully deleted
I20260618 22:37:29.098086 4749 ts_tablet_manager.cc:1939] T f232d39cbabc43829ba7ee051f315bf3 P 9a45b48787cc4363bb39deee0146be88: Deleting consensus metadata
I20260618 22:37:29.099813 4243 catalog_manager.cc:4841] TS 9a45b48787cc4363bb39deee0146be88 (127.18.150.131:33987): tablet f232d39cbabc43829ba7ee051f315bf3 (table test-table [id=0c2064d0d8854d51878320482c6515f4]) successfully deleted
I20260618 22:37:29.179322 4750 tablet_replica.cc:330] T f232d39cbabc43829ba7ee051f315bf3 P a48d3456b54f432d8dbeb78e993facfa: stopping tablet replica
I20260618 22:37:29.179517 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 4318
W20260618 22:37:29.200026 4245 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.150.129:45839 (error 108)
I20260618 22:37:29.200294 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 4449
I20260618 22:37:29.202081 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:29.201578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:45839 (local address 127.0.0.1:59928)
0618 22:37:29.201754 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:29.201887 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:45839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260618 22:37:29.202379 4245 catalog_manager.cc:4584] TS a48d3456b54f432d8dbeb78e993facfa (127.18.150.129:45839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f232d39cbabc43829ba7ee051f315bf3: Network error: Client connection negotiation failed: client connection to 127.18.150.129:45839: connect: Connection refused (error 111)
I20260618 22:37:29.208706 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:29.201375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57542 (local address 127.18.150.130:34371)
0618 22:37:29.201536 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:29.201541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:29.201560 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:37:29.201770 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:29.201776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:29.201918 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:29.202016 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:29.202729 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:29.203887 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:29.205183 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:29.205854 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:29.206158 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:29.207048 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:29.207060 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:29.207080 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:29.207880 (+ 800us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:29.207941 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:29.207966 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:29.208075 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:29.208141 (+ 66us) server_negotiation.cc:299] Negotiation successful
0618 22:37:29.208240 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:37:29.209394 4571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:29.256126 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:29.255570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:45839 (local address 127.0.0.1:59934)
0618 22:37:29.255817 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:29.255934 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:45839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260618 22:37:29.257118 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:29.256739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:45839 (local address 127.0.0.1:59938)
0618 22:37:29.256890 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:29.256974 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:45839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260618 22:37:29.317382 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 4449
I20260618 22:37:29.317632 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:29.317059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:45839 (local address 127.0.0.1:59946)
0618 22:37:29.317364 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:29.317465 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:45839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260618 22:37:29.318652 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:29.318314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:45839 (local address 127.0.0.1:59960)
0618 22:37:29.318462 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:29.318543 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:45839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260618 22:37:29.332196 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 4579
I20260618 22:37:29.338747 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:29.332879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44380 (local address 127.18.150.131:33987)
0618 22:37:29.333089 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:29.333094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:29.333114 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:37:29.333294 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:29.333300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:29.333385 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:29.333468 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:29.333857 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:29.334670 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:29.335973 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:29.336439 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:29.336569 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:29.337479 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:29.337488 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:29.337503 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:29.338106 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:29.338163 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:29.338182 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:29.338262 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:29.338375 (+ 113us) server_negotiation.cc:299] Negotiation successful
0618 22:37:29.338460 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260618 22:37:29.339387 4701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:29.399355 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:29.398873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:45839 (local address 127.0.0.1:59964)
0618 22:37:29.399069 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:29.399200 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:45839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260618 22:37:29.400424 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:29.400034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:45839 (local address 127.0.0.1:59970)
0618 22:37:29.400236 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:29.400308 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:45839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260618 22:37:29.438452 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 4579
I20260618 22:37:29.451470 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 4228
I20260618 22:37:29.452502 4289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:29.560186 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 4228
2026-06-18T22:37:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2701 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:29 dist-test-slave-h7jz krb5kdc[4759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:29 dist-test-slave-h7jz krb5kdc[4759](info): set up 2 sockets
Jun 18 22:37:29 dist-test-slave-h7jz krb5kdc[4759](info): commencing operation
krb5kdc: starting...
W20260618 22:37:30.581727 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:30 dist-test-slave-h7jz krb5kdc[4759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:37:30.803249 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39767
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:39767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:31.116420 4772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:31.116693 4772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:31.116784 4772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:31.124042 4772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:31.124157 4772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:31.124203 4772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:31.124257 4772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:31.124298 4772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:31.134003 4772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:39767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:39767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:31.135221 4772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:31.136449 4772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:31.147471 4777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:31.147506 4780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:31.147536 4778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:31.148101 4772 server_base.cc:1034] running on GCE node
I20260618 22:37:31.148928 4772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:31.150393 4772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:31.151592 4772 hybrid_clock.cc:648] HybridClock initialized: now 1781822251151544 us; error 43 us; skew 500 ppm
I20260618 22:37:31.151924 4772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:31 dist-test-slave-h7jz krb5kdc[4759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822251, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:31.154963 4772 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:37:31.156289 4772 webserver.cc:460] Webserver started at http://127.18.150.190:42505/ using document root <none> and password file <none>
I20260618 22:37:31.156931 4772 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:31.157028 4772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:31.157263 4772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:31.160207 4772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "d15b51c0d90b44579eb7f83d991eb95a"
format_stamp: "Formatted at 2026-06-18 22:37:31 on dist-test-slave-h7jz"
server_key: "5dd23a890bcbcaa0623ea61f5c8b2272"
server_key_iv: "6d649d902ca1c15542922a7024d3287c"
server_key_version: "encryptionkey@0"
I20260618 22:37:31.160939 4772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "d15b51c0d90b44579eb7f83d991eb95a"
format_stamp: "Formatted at 2026-06-18 22:37:31 on dist-test-slave-h7jz"
server_key: "5dd23a890bcbcaa0623ea61f5c8b2272"
server_key_iv: "6d649d902ca1c15542922a7024d3287c"
server_key_version: "encryptionkey@0"
I20260618 22:37:31.165971 4772 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260618 22:37:31.169303 4787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:31.170642 4772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:37:31.170814 4772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "d15b51c0d90b44579eb7f83d991eb95a"
format_stamp: "Formatted at 2026-06-18 22:37:31 on dist-test-slave-h7jz"
server_key: "5dd23a890bcbcaa0623ea61f5c8b2272"
server_key_iv: "6d649d902ca1c15542922a7024d3287c"
server_key_version: "encryptionkey@0"
I20260618 22:37:31.170991 4772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:31.192229 4772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:31.196139 4772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:31.196398 4772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:31.212747 4772 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:39767
I20260618 22:37:31.212760 4838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:39767 every 8 connection(s)
I20260618 22:37:31.214040 4772 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:31.214279 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 4772
I20260618 22:37:31.214428 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:31.214910 19034 external_mini_cluster.cc:1428] Setting key 77f810a321e1e08a48148c3576a10858
I20260618 22:37:31.219224 4839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 18 22:37:31 dist-test-slave-h7jz krb5kdc[4759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:31.230755 4839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a: Bootstrap starting.
I20260618 22:37:31.234534 4839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:31.235684 4839 log.cc:824] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:31.237922 4842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:31.216503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39418 (local address 127.18.150.190:39767)
0618 22:37:31.217270 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:31.217285 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:31.217333 (+ 48us) server_negotiation.cc:407] Connection header received
0618 22:37:31.219564 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:31.219591 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:31.220050 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:31.220473 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:31.221713 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:31.223635 (+ 1922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:31.224967 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:31.225930 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:31.226122 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:31.229870 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:31.229880 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:31.229899 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:31.229962 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:31.233619 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:31.234348 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:31.234356 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:31.234363 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:31.234566 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:31.235006 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:31.235011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:31.235014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:31.235667 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:31.235943 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:31.236398 (+ 455us) server_negotiation.cc:299] Negotiation successful
0618 22:37:31.236746 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":326,"threads_started":1}
I20260618 22:37:31.238880 4839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a: No bootstrap required, opened a new log
I20260618 22:37:31.245899 4839 raft_consensus.cc:348] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d15b51c0d90b44579eb7f83d991eb95a" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39767 } }
I20260618 22:37:31.246302 4839 raft_consensus.cc:374] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:31.246425 4839 raft_consensus.cc:729] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d15b51c0d90b44579eb7f83d991eb95a, State: Initialized, Role: FOLLOWER
I20260618 22:37:31.247001 4839 consensus_queue.cc:260] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d15b51c0d90b44579eb7f83d991eb95a" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39767 } }
I20260618 22:37:31.247200 4839 raft_consensus.cc:388] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:31.247316 4839 raft_consensus.cc:482] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:31.247458 4839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:31.249459 4839 raft_consensus.cc:504] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d15b51c0d90b44579eb7f83d991eb95a" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39767 } }
I20260618 22:37:31.249917 4839 leader_election.cc:302] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d15b51c0d90b44579eb7f83d991eb95a; no voters:
I20260618 22:37:31.250475 4839 leader_election.cc:288] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:31.250622 4844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:31.251636 4844 raft_consensus.cc:686] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [term 1 LEADER]: Becoming Leader. State: Replica: d15b51c0d90b44579eb7f83d991eb95a, State: Running, Role: LEADER
I20260618 22:37:31.252220 4844 consensus_queue.cc:237] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d15b51c0d90b44579eb7f83d991eb95a" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39767 } }
I20260618 22:37:31.252519 4839 sys_catalog.cc:564] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:31.254879 4846 sys_catalog.cc:455] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d15b51c0d90b44579eb7f83d991eb95a. Latest consensus state: current_term: 1 leader_uuid: "d15b51c0d90b44579eb7f83d991eb95a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d15b51c0d90b44579eb7f83d991eb95a" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39767 } } }
I20260618 22:37:31.255221 4846 sys_catalog.cc:458] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:31.255722 4845 sys_catalog.cc:455] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d15b51c0d90b44579eb7f83d991eb95a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d15b51c0d90b44579eb7f83d991eb95a" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 39767 } } }
I20260618 22:37:31.256016 4845 sys_catalog.cc:458] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:31.256747 4852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:31.260993 4852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:31.275840 4852 catalog_manager.cc:1347] Generated new cluster ID: 18a6872d201b467481a3d6b82ebfc942
I20260618 22:37:31.275946 4852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:31.293292 4852 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:31.295270 4852 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:31.305212 4852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a: Generated new TSK 0
I20260618 22:37:31.306100 4852 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:37:31.373150 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39767
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:31.692389 4865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:31.692654 4865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:31.692747 4865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:31.700491 4865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:31.700605 4865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:31.700717 4865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:37:31.710421 4865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:31.711620 4865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:31.712889 4865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:31.726207 4871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:31.726197 4873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:31.727353 4865 server_base.cc:1034] running on GCE node
W20260618 22:37:31.729099 4870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:31.729897 4865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:31.731176 4865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:31.732405 4865 hybrid_clock.cc:648] HybridClock initialized: now 1781822251732317 us; error 63 us; skew 500 ppm
I20260618 22:37:31.732755 4865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:31 dist-test-slave-h7jz krb5kdc[4759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822251, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:31.736119 4865 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:37:31.737533 4865 webserver.cc:460] Webserver started at http://127.18.150.129:35841/ using document root <none> and password file <none>
I20260618 22:37:31.738265 4865 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:31.738376 4865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:31.738667 4865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:31.741456 4865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "bfac4bf94b54447e8963ccbdcc265dc9"
format_stamp: "Formatted at 2026-06-18 22:37:31 on dist-test-slave-h7jz"
server_key: "d8ee34cd890477b511e3034afbd9b3f0"
server_key_iv: "e86b99dc7eaf52e869131cc6dbbf724a"
server_key_version: "encryptionkey@0"
I20260618 22:37:31.742216 4865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "bfac4bf94b54447e8963ccbdcc265dc9"
format_stamp: "Formatted at 2026-06-18 22:37:31 on dist-test-slave-h7jz"
server_key: "d8ee34cd890477b511e3034afbd9b3f0"
server_key_iv: "e86b99dc7eaf52e869131cc6dbbf724a"
server_key_version: "encryptionkey@0"
I20260618 22:37:31.747306 4865 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:37:31.750638 4880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:31.751900 4865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:37:31.752032 4865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "bfac4bf94b54447e8963ccbdcc265dc9"
format_stamp: "Formatted at 2026-06-18 22:37:31 on dist-test-slave-h7jz"
server_key: "d8ee34cd890477b511e3034afbd9b3f0"
server_key_iv: "e86b99dc7eaf52e869131cc6dbbf724a"
server_key_version: "encryptionkey@0"
I20260618 22:37:31.752245 4865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:31.767438 4865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:31.771385 4865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:31.771636 4865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:31.772517 4865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:31.773734 4865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:31.773810 4865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:31.773917 4865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:31.773994 4865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260618 22:37:31.803741 4865 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:41761
I20260618 22:37:31.803779 4992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:41761 every 8 connection(s)
I20260618 22:37:31.805227 4865 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:37:31.808102 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 4865
I20260618 22:37:31.808279 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:37:31.808863 19034 external_mini_cluster.cc:1428] Setting key f2c41ee7a32e5d9f3bc92960d1f399da
Jun 18 22:37:31 dist-test-slave-h7jz krb5kdc[4759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822251, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:31.824718 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:31.807801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:35925 (local address 127.18.150.190:39767)
0618 22:37:31.808203 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:31.808209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:31.809443 (+ 1234us) server_negotiation.cc:407] Connection header received
0618 22:37:31.811477 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:31.811482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:31.811563 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:31.811653 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:31.813200 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:31.813904 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:31.815429 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:31.815839 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:31.815952 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:31.819410 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:31.819415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:31.819418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:31.819451 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:31.821816 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:31.822527 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:31.822533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:31.822536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:31.822653 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:31.823056 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:31.823059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:31.823061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:31.823311 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:31.823456 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:31.824133 (+ 677us) server_negotiation.cc:299] Negotiation successful
0618 22:37:31.824363 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":193,"threads_started":1}
I20260618 22:37:31.825686 4996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:31.808399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39767 (local address 127.18.150.129:35925)
0618 22:37:31.809282 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:31.809322 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:31.811205 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:31.811905 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:31.811913 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:31.812439 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:31.813033 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:31.813051 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:31.814062 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:31.814066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:31.815291 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:31.815303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:31.816049 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:31.816056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:31.816299 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:31.816967 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:37:31.816989 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:31.819190 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:31.821963 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:31.821980 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:31.821998 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:31.822418 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:31.822752 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:31.822757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:31.822762 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:31.822948 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:31.823431 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:31.823438 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:31.823772 (+ 334us) client_negotiation.cc:769] Sending connection context
0618 22:37:31.824005 (+ 233us) client_negotiation.cc:240] Negotiation successful
0618 22:37:31.824528 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":376,"threads_started":1}
I20260618 22:37:31.827240 4993 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39767
I20260618 22:37:31.827592 4993 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:31.828512 4993 heartbeater.cc:511] Master 127.18.150.190:39767 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
I20260618 22:37:31.830932 4804 ts_manager.cc:194] Registered new tserver with Master: bfac4bf94b54447e8963ccbdcc265dc9 (127.18.150.129:41761)
I20260618 22:37:31.832353 4804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:35925
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:37:31.857914 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39767
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:32.164615 4999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:32.164897 4999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:32.165022 4999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:32.172302 4999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:32.172403 4999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:32.172552 4999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:37:32.182652 4999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:32.183924 4999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:32.185202 4999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:32.197865 5004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:32.197980 5005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:32.197932 5007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:32.198559 4999 server_base.cc:1034] running on GCE node
I20260618 22:37:32.199096 4999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:32.199928 4999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:32.201133 4999 hybrid_clock.cc:648] HybridClock initialized: now 1781822252201070 us; error 53 us; skew 500 ppm
I20260618 22:37:32.201426 4999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:32 dist-test-slave-h7jz krb5kdc[4759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822252, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:32.204367 4999 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:37:32.205650 4999 webserver.cc:460] Webserver started at http://127.18.150.130:40085/ using document root <none> and password file <none>
I20260618 22:37:32.206314 4999 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:32.206419 4999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:32.206705 4999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:32.209499 4999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "5367243362584706b6e016ad251540ef"
format_stamp: "Formatted at 2026-06-18 22:37:32 on dist-test-slave-h7jz"
server_key: "9fca302f54b649f97746ccc764bdc6ed"
server_key_iv: "ecb820fba678d1864341f9c096a8b985"
server_key_version: "encryptionkey@0"
I20260618 22:37:32.210227 4999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "5367243362584706b6e016ad251540ef"
format_stamp: "Formatted at 2026-06-18 22:37:32 on dist-test-slave-h7jz"
server_key: "9fca302f54b649f97746ccc764bdc6ed"
server_key_iv: "ecb820fba678d1864341f9c096a8b985"
server_key_version: "encryptionkey@0"
I20260618 22:37:32.215166 4999 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:37:32.218554 5014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:32.220053 4999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:37:32.220198 4999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "5367243362584706b6e016ad251540ef"
format_stamp: "Formatted at 2026-06-18 22:37:32 on dist-test-slave-h7jz"
server_key: "9fca302f54b649f97746ccc764bdc6ed"
server_key_iv: "ecb820fba678d1864341f9c096a8b985"
server_key_version: "encryptionkey@0"
I20260618 22:37:32.220382 4999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:32.253052 4999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:32.257062 4999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:32.257319 4999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:32.258314 4999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:32.259698 4999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:32.259783 4999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:32.259904 4999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:32.259955 4999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:32.292594 4999 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:39347
I20260618 22:37:32.292637 5126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:39347 every 8 connection(s)
I20260618 22:37:32.293860 4999 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:37:32.299578 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 4999
I20260618 22:37:32.299734 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:37:32.300148 19034 external_mini_cluster.cc:1428] Setting key b5e01a057e9c63d35d6ce6ed4e97ecc7
Jun 18 22:37:32 dist-test-slave-h7jz krb5kdc[4759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822252, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:32.313153 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:32.296365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:52889 (local address 127.18.150.190:39767)
0618 22:37:32.296528 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:32.296534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:32.297669 (+ 1135us) server_negotiation.cc:407] Connection header received
0618 22:37:32.299492 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:32.299498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:32.299585 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:32.299691 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:32.301335 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.302212 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.303712 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.304124 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.304230 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:32.307618 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:32.307623 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:32.307626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:32.307665 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:32.310374 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:32.311122 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:32.311126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:32.311129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:32.311256 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:32.311660 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:32.311663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:32.311665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:32.311925 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:32.312031 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:32.312750 (+ 719us) server_negotiation.cc:299] Negotiation successful
0618 22:37:32.312916 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:37:32.314468 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:32.296800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39767 (local address 127.18.150.130:52889)
0618 22:37:32.297483 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:32.297520 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:32.299214 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:32.299972 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:32.299980 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:32.300518 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:32.301172 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:32.301190 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.302347 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.302351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:32.303578 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:32.303589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.304338 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.304345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:32.304564 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:32.305241 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:37:32.305279 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:32.307417 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:32.310493 (+ 3076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:32.310500 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:32.310516 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:32.311014 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:32.311349 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:32.311354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:32.311358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:32.311573 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:32.312026 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:32.312033 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:32.312433 (+ 400us) client_negotiation.cc:769] Sending connection context
0618 22:37:32.312705 (+ 272us) client_negotiation.cc:240] Negotiation successful
0618 22:37:32.313183 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":288,"threads_started":1}
I20260618 22:37:32.315806 5127 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39767
I20260618 22:37:32.316107 5127 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:32.316915 5127 heartbeater.cc:511] Master 127.18.150.190:39767 requested a full tablet report, sending...
I20260618 22:37:32.318467 4804 ts_manager.cc:194] Registered new tserver with Master: 5367243362584706b6e016ad251540ef (127.18.150.130:39347)
I20260618 22:37:32.319250 4804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:52889
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:37:32.352686 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39767
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:35071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:32.667963 5132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:32.668337 5132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:32.668490 5132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:32.675843 5132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:32.675958 5132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:32.676096 5132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:37:32.685977 5132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:35071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:39767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:32.687253 5132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:32.688578 5132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:32.702375 5140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:32.702451 5137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:32.702451 5138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:32.703063 5132 server_base.cc:1034] running on GCE node
I20260618 22:37:32.703667 5132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:32.704617 5132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:32.705840 5132 hybrid_clock.cc:648] HybridClock initialized: now 1781822252705787 us; error 47 us; skew 500 ppm
I20260618 22:37:32.706233 5132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:32 dist-test-slave-h7jz krb5kdc[4759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822252, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:32.709223 5132 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:37:32.710583 5132 webserver.cc:460] Webserver started at http://127.18.150.131:35457/ using document root <none> and password file <none>
I20260618 22:37:32.711246 5132 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:32.711354 5132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:32.711637 5132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:32.714505 5132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "b34afabb09f24c358952a21dee40a78c"
format_stamp: "Formatted at 2026-06-18 22:37:32 on dist-test-slave-h7jz"
server_key: "e7be2a1a29328572ca401c5901dc0401"
server_key_iv: "ea32e3a659f19843a3ec4209d041bc1d"
server_key_version: "encryptionkey@0"
I20260618 22:37:32.715226 5132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "b34afabb09f24c358952a21dee40a78c"
format_stamp: "Formatted at 2026-06-18 22:37:32 on dist-test-slave-h7jz"
server_key: "e7be2a1a29328572ca401c5901dc0401"
server_key_iv: "ea32e3a659f19843a3ec4209d041bc1d"
server_key_version: "encryptionkey@0"
I20260618 22:37:32.720141 5132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260618 22:37:32.723608 5147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:32.725040 5132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260618 22:37:32.725184 5132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "b34afabb09f24c358952a21dee40a78c"
format_stamp: "Formatted at 2026-06-18 22:37:32 on dist-test-slave-h7jz"
server_key: "e7be2a1a29328572ca401c5901dc0401"
server_key_iv: "ea32e3a659f19843a3ec4209d041bc1d"
server_key_version: "encryptionkey@0"
I20260618 22:37:32.725358 5132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:32.802181 5132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:32.807555 5132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:32.807850 5132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:32.808979 5132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:32.810457 5132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:32.810532 5132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:32.810616 5132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:32.810658 5132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:32.835448 4993 heartbeater.cc:503] Master 127.18.150.190:39767 was elected leader, sending a full tablet report...
I20260618 22:37:32.840107 5132 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:37643
I20260618 22:37:32.840147 5259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:37643 every 8 connection(s)
I20260618 22:37:32.841360 5132 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:37:32.844662 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 5132
I20260618 22:37:32.844877 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:37:32.845419 19034 external_mini_cluster.cc:1428] Setting key cd9400300318af58e06a36732bf62e2b
Jun 18 22:37:32 dist-test-slave-h7jz krb5kdc[4759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822252, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:32.863427 5262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:32.843666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:58343 (local address 127.18.150.190:39767)
0618 22:37:32.844115 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:32.844120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:32.845227 (+ 1107us) server_negotiation.cc:407] Connection header received
0618 22:37:32.847481 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:32.847486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:32.847565 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:32.847674 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:32.849168 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.849932 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.851387 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.851783 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.851873 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:32.858191 (+ 6318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:32.858198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:32.858231 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:32.858272 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:32.860943 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:32.861557 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:32.861561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:32.861564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:32.861670 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:32.862025 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:32.862028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:32.862030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:32.862284 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:32.862388 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:32.862976 (+ 588us) server_negotiation.cc:299] Negotiation successful
0618 22:37:32.863130 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":220,"threads_started":1}
I20260618 22:37:32.864318 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:32.844270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:39767 (local address 127.18.150.131:58343)
0618 22:37:32.845063 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:32.845119 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:32.847238 (+ 2119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:32.847886 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:32.847898 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:32.848465 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:32.849026 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:32.849043 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.850070 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.850073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:32.851269 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:32.851282 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.851939 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.851943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:32.852130 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:32.855820 (+ 3690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:37:32.855843 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:32.857992 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:32.861063 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:32.861069 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:32.861082 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:32.861475 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:32.861743 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:32.861746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:32.861748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:32.861915 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:32.862374 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:32.862380 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:32.862678 (+ 298us) client_negotiation.cc:769] Sending connection context
0618 22:37:32.862902 (+ 224us) client_negotiation.cc:240] Negotiation successful
0618 22:37:32.863256 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":282,"threads_started":1}
I20260618 22:37:32.865571 5260 heartbeater.cc:347] Connected to a master server at 127.18.150.190:39767
I20260618 22:37:32.865823 5260 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:32.866496 5260 heartbeater.cc:511] Master 127.18.150.190:39767 requested a full tablet report, sending...
I20260618 22:37:32.867780 4804 ts_manager.cc:194] Registered new tserver with Master: b34afabb09f24c358952a21dee40a78c (127.18.150.131:37643)
I20260618 22:37:32.868563 4804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:58343
I20260618 22:37:32.872148 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:37:32.887209 5262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:32.875896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.18.150.190:39767)
0618 22:37:32.876030 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:32.876034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:32.876474 (+ 440us) server_negotiation.cc:407] Connection header received
0618 22:37:32.876613 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:32.876617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:32.876673 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:32.876740 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:32.877924 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.878658 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.879928 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.880297 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.880384 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:32.882764 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:32.882771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:32.882775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:32.882804 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:32.885111 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:32.885704 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:32.885707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:32.885710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:32.885811 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:32.886142 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:32.886145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:32.886147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:32.886430 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:32.886608 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:32.886859 (+ 251us) server_negotiation.cc:299] Negotiation successful
0618 22:37:32.886985 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:37:32.890581 4804 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:37:32.897009 4804 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:37:32.928589 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:32.909333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39364 (local address 127.18.150.130:39347)
0618 22:37:32.918733 (+ 9400us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:32.918743 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:32.920014 (+ 1271us) server_negotiation.cc:407] Connection header received
0618 22:37:32.920150 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:32.920160 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:32.920654 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:32.920860 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:32.921836 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.923208 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.925688 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.927175 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.927328 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:32.927513 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:32.928085 (+ 572us) server_negotiation.cc:299] Negotiation successful
0618 22:37:32.928263 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9201,"thread_start_us":9061,"threads_started":1}
I20260618 22:37:32.928650 5277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:32.908902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:39347 (local address 127.0.0.1:39364)
0618 22:37:32.919333 (+ 10431us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:32.919389 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:32.919598 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:32.921135 (+ 1537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:32.921145 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:32.921182 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:32.921626 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:32.921639 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.923362 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.923372 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:32.925525 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:32.925543 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.927635 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.927642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:32.927831 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:32.927882 (+ 51us) client_negotiation.cc:769] Sending connection context
0618 22:37:32.928018 (+ 136us) client_negotiation.cc:240] Negotiation successful
0618 22:37:32.928201 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10133,"thread_start_us":10852,"threads_started":1}
I20260618 22:37:32.929220 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:32.908916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:37643 (local address 127.0.0.1:42168)
0618 22:37:32.920065 (+ 11149us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:32.920084 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:32.920211 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:32.922401 (+ 2190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:32.922406 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:32.922429 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:32.922836 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:32.922850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.925104 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.925109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:32.926790 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:32.926805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.928451 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.928457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:32.928640 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:32.928672 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:37:32.928740 (+ 68us) client_negotiation.cc:240] Negotiation successful
0618 22:37:32.928848 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11022,"mutex_wait_us":176,"thread_start_us":12700,"threads_started":1}
I20260618 22:37:32.930222 5275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:32.909760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41761 (local address 127.0.0.1:48880)
0618 22:37:32.922054 (+ 12294us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:32.922070 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:32.922251 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:32.923124 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:32.923128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:32.923151 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:32.923632 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:32.923648 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.925403 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.925412 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:32.927426 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:32.927439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.929382 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.929389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:32.929581 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:32.929603 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:37:32.929677 (+ 74us) client_negotiation.cc:240] Negotiation successful
0618 22:37:32.929902 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12161,"thread_start_us":9144,"threads_started":1}
I20260618 22:37:32.930372 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:32.908984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42168 (local address 127.18.150.131:37643)
0618 22:37:32.921736 (+ 12752us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:32.921744 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:32.921778 (+ 34us) server_negotiation.cc:407] Connection header received
0618 22:37:32.921871 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:32.921878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:32.922194 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:32.923027 (+ 833us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:32.923318 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.924982 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.926973 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.928238 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.929695 (+ 1457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:32.929847 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:32.929909 (+ 62us) server_negotiation.cc:299] Negotiation successful
0618 22:37:32.930094 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12601,"thread_start_us":12455,"threads_started":1}
I20260618 22:37:32.930781 5278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:32.910131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48880 (local address 127.18.150.129:41761)
0618 22:37:32.921249 (+ 11118us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:32.921259 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:32.922157 (+ 898us) server_negotiation.cc:407] Connection header received
0618 22:37:32.922427 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:32.922435 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:32.922672 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:32.922848 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:32.923830 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.925283 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.927588 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:32.929016 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:32.929168 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:32.929344 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:32.929789 (+ 445us) server_negotiation.cc:299] Negotiation successful
0618 22:37:32.929963 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10929,"thread_start_us":10776,"threads_started":1}
I20260618 22:37:32.932914 5062 tablet_service.cc:1467] Processing CreateTablet for tablet 5a9e801ce4564c0db1934fa36588eddf (DEFAULT_TABLE table=test-table [id=971493c2a1954a8ab1a58c73997bc4e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:32.933882 5195 tablet_service.cc:1467] Processing CreateTablet for tablet 5a9e801ce4564c0db1934fa36588eddf (DEFAULT_TABLE table=test-table [id=971493c2a1954a8ab1a58c73997bc4e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:32.934410 5062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9e801ce4564c0db1934fa36588eddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:32.934247 4928 tablet_service.cc:1467] Processing CreateTablet for tablet 5a9e801ce4564c0db1934fa36588eddf (DEFAULT_TABLE table=test-table [id=971493c2a1954a8ab1a58c73997bc4e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:32.935079 5195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9e801ce4564c0db1934fa36588eddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:32.935545 4928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9e801ce4564c0db1934fa36588eddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:32.946291 5279 tablet_bootstrap.cc:492] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: Bootstrap starting.
I20260618 22:37:32.948670 5281 tablet_bootstrap.cc:492] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: Bootstrap starting.
I20260618 22:37:32.950182 5280 tablet_bootstrap.cc:492] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: Bootstrap starting.
I20260618 22:37:32.951079 5279 tablet_bootstrap.cc:654] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:32.952116 5281 tablet_bootstrap.cc:654] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:32.952347 5279 log.cc:824] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:32.952948 5280 tablet_bootstrap.cc:654] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:32.953356 5281 log.cc:824] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:32.953915 5280 log.cc:824] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:32.956156 5281 tablet_bootstrap.cc:492] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: No bootstrap required, opened a new log
I20260618 22:37:32.956413 5281 ts_tablet_manager.cc:1397] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260618 22:37:32.956590 5280 tablet_bootstrap.cc:492] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: No bootstrap required, opened a new log
I20260618 22:37:32.956842 5280 ts_tablet_manager.cc:1397] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260618 22:37:32.958213 5279 tablet_bootstrap.cc:492] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: No bootstrap required, opened a new log
I20260618 22:37:32.958473 5279 ts_tablet_manager.cc:1397] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260618 22:37:32.963761 5281 raft_consensus.cc:348] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34afabb09f24c358952a21dee40a78c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37643 } } peers { permanent_uuid: "5367243362584706b6e016ad251540ef" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39347 } } peers { permanent_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41761 } }
I20260618 22:37:32.964099 5281 raft_consensus.cc:374] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:32.964200 5281 raft_consensus.cc:729] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfac4bf94b54447e8963ccbdcc265dc9, State: Initialized, Role: FOLLOWER
I20260618 22:37:32.964740 5281 consensus_queue.cc:260] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34afabb09f24c358952a21dee40a78c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37643 } } peers { permanent_uuid: "5367243362584706b6e016ad251540ef" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39347 } } peers { permanent_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41761 } }
I20260618 22:37:32.966380 5281 ts_tablet_manager.cc:1428] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: Time spent starting tablet: real 0.010s user 0.009s sys 0.001s
I20260618 22:37:32.966913 5280 raft_consensus.cc:348] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34afabb09f24c358952a21dee40a78c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37643 } } peers { permanent_uuid: "5367243362584706b6e016ad251540ef" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39347 } } peers { permanent_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41761 } }
I20260618 22:37:32.967317 5280 raft_consensus.cc:374] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:32.967412 5280 raft_consensus.cc:729] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5367243362584706b6e016ad251540ef, State: Initialized, Role: FOLLOWER
I20260618 22:37:32.967967 5280 consensus_queue.cc:260] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34afabb09f24c358952a21dee40a78c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37643 } } peers { permanent_uuid: "5367243362584706b6e016ad251540ef" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39347 } } peers { permanent_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41761 } }
I20260618 22:37:32.968235 5279 raft_consensus.cc:348] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34afabb09f24c358952a21dee40a78c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37643 } } peers { permanent_uuid: "5367243362584706b6e016ad251540ef" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39347 } } peers { permanent_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41761 } }
I20260618 22:37:32.968672 5279 raft_consensus.cc:374] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:32.968765 5279 raft_consensus.cc:729] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b34afabb09f24c358952a21dee40a78c, State: Initialized, Role: FOLLOWER
I20260618 22:37:32.969410 5279 consensus_queue.cc:260] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34afabb09f24c358952a21dee40a78c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37643 } } peers { permanent_uuid: "5367243362584706b6e016ad251540ef" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39347 } } peers { permanent_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41761 } }
I20260618 22:37:32.969588 5280 ts_tablet_manager.cc:1428] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260618 22:37:32.970115 5127 heartbeater.cc:503] Master 127.18.150.190:39767 was elected leader, sending a full tablet report...
I20260618 22:37:32.971084 5260 heartbeater.cc:503] Master 127.18.150.190:39767 was elected leader, sending a full tablet report...
I20260618 22:37:32.972023 5279 ts_tablet_manager.cc:1428] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260618 22:37:33.031082 5285 raft_consensus.cc:482] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:37:33.031355 5285 raft_consensus.cc:504] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34afabb09f24c358952a21dee40a78c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37643 } } peers { permanent_uuid: "5367243362584706b6e016ad251540ef" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39347 } } peers { permanent_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41761 } }
I20260618 22:37:33.033015 5285 leader_election.cc:288] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b34afabb09f24c358952a21dee40a78c (127.18.150.131:37643), 5367243362584706b6e016ad251540ef (127.18.150.130:39347)
I20260618 22:37:33.039350 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:33.033357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:37263 (local address 127.18.150.131:37643)
0618 22:37:33.033621 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:33.033629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:33.034418 (+ 789us) server_negotiation.cc:407] Connection header received
0618 22:37:33.034594 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:33.034597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:33.034656 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:33.034724 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:33.035345 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.036005 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.037750 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.038520 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.038661 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:33.038734 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:33.038989 (+ 255us) server_negotiation.cc:299] Negotiation successful
0618 22:37:33.039108 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260618 22:37:33.039433 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:33.033521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:44991 (local address 127.18.150.130:39347)
0618 22:37:33.033729 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:33.033735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:33.034334 (+ 599us) server_negotiation.cc:407] Connection header received
0618 22:37:33.034519 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:33.034523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:33.034583 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:33.034660 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:33.035266 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.035959 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.037786 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.038585 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.038729 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:33.038800 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:33.039075 (+ 275us) server_negotiation.cc:299] Negotiation successful
0618 22:37:33.039197 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:37:33.039601 5288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:33.033439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:37643 (local address 127.18.150.129:37263)
0618 22:37:33.034311 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:33.034333 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:33.034465 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:33.034756 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:33.034761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:33.034787 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:33.035201 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:33.035212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.036128 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.036132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:33.037606 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:33.037618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.038712 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.038719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:33.038848 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:33.038874 (+ 26us) client_negotiation.cc:769] Sending connection context
0618 22:37:33.038954 (+ 80us) client_negotiation.cc:240] Negotiation successful
0618 22:37:33.039070 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":20,"spinlock_wait_cycles":896,"thread_start_us":396,"threads_started":1}
I20260618 22:37:33.039753 5289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:33.033321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:39347 (local address 127.18.150.129:44991)
0618 22:37:33.034222 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:33.034239 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:33.034368 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:33.034691 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:33.034695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:33.034718 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:33.035121 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:33.035135 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.036082 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.036086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:33.037637 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:33.037649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.038792 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.038798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:33.038935 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:33.038957 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:37:33.039034 (+ 77us) client_negotiation.cc:240] Negotiation successful
0618 22:37:33.039147 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":62,"spinlock_wait_cycles":24832,"thread_start_us":643,"threads_started":1}
I20260618 22:37:33.040565 5082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9e801ce4564c0db1934fa36588eddf" candidate_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5367243362584706b6e016ad251540ef" is_pre_election: true
I20260618 22:37:33.040539 5215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9e801ce4564c0db1934fa36588eddf" candidate_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b34afabb09f24c358952a21dee40a78c" is_pre_election: true
I20260618 22:37:33.041088 5215 raft_consensus.cc:2454] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfac4bf94b54447e8963ccbdcc265dc9 in term 0.
I20260618 22:37:33.041088 5082 raft_consensus.cc:2454] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfac4bf94b54447e8963ccbdcc265dc9 in term 0.
I20260618 22:37:33.041739 4882 leader_election.cc:302] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5367243362584706b6e016ad251540ef, bfac4bf94b54447e8963ccbdcc265dc9; no voters:
I20260618 22:37:33.042158 5285 raft_consensus.cc:2781] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:37:33.042297 5285 raft_consensus.cc:482] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:37:33.042385 5285 raft_consensus.cc:3037] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:33.044668 5285 raft_consensus.cc:504] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34afabb09f24c358952a21dee40a78c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37643 } } peers { permanent_uuid: "5367243362584706b6e016ad251540ef" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39347 } } peers { permanent_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41761 } }
I20260618 22:37:33.045238 5285 leader_election.cc:288] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [CANDIDATE]: Term 1 election: Requested vote from peers b34afabb09f24c358952a21dee40a78c (127.18.150.131:37643), 5367243362584706b6e016ad251540ef (127.18.150.130:39347)
I20260618 22:37:33.045715 5082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9e801ce4564c0db1934fa36588eddf" candidate_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5367243362584706b6e016ad251540ef"
I20260618 22:37:33.045739 5215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9e801ce4564c0db1934fa36588eddf" candidate_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b34afabb09f24c358952a21dee40a78c"
W20260618 22:37:33.045756 5128 tablet.cc:2369] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:37:33.045959 5215 raft_consensus.cc:3037] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:33.045971 5082 raft_consensus.cc:3037] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:33.048094 5082 raft_consensus.cc:2454] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfac4bf94b54447e8963ccbdcc265dc9 in term 1.
I20260618 22:37:33.048202 5215 raft_consensus.cc:2454] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfac4bf94b54447e8963ccbdcc265dc9 in term 1.
I20260618 22:37:33.048445 4882 leader_election.cc:302] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5367243362584706b6e016ad251540ef, bfac4bf94b54447e8963ccbdcc265dc9; no voters:
I20260618 22:37:33.048689 5285 raft_consensus.cc:2781] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:33.049093 5285 raft_consensus.cc:686] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 1 LEADER]: Becoming Leader. State: Replica: bfac4bf94b54447e8963ccbdcc265dc9, State: Running, Role: LEADER
I20260618 22:37:33.049528 5285 consensus_queue.cc:237] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34afabb09f24c358952a21dee40a78c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37643 } } peers { permanent_uuid: "5367243362584706b6e016ad251540ef" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39347 } } peers { permanent_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41761 } }
I20260618 22:37:33.053993 4804 catalog_manager.cc:5495] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfac4bf94b54447e8963ccbdcc265dc9 (127.18.150.129). New cstate: current_term: 1 leader_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34afabb09f24c358952a21dee40a78c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5367243362584706b6e016ad251540ef" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfac4bf94b54447e8963ccbdcc265dc9" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41761 } health_report { overall_health: HEALTHY } } }
W20260618 22:37:33.057329 4994 tablet.cc:2369] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:37:33.085516 5278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:33.080455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48888 (local address 127.18.150.129:41761)
0618 22:37:33.080625 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:33.080629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:33.080720 (+ 91us) server_negotiation.cc:407] Connection header received
0618 22:37:33.080884 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:33.080887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:33.080963 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:33.081086 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:37:33.081401 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.082153 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.083529 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.083913 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.083998 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:33.084496 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:37:33.084999 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:37:33.085125 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:33.085281 (+ 156us) server_negotiation.cc:299] Negotiation successful
0618 22:37:33.085321 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260618 22:37:33.092900 5261 tablet.cc:2369] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:37:33.093719 5215 raft_consensus.cc:1261] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c [term 1 FOLLOWER]: Refusing update from remote peer bfac4bf94b54447e8963ccbdcc265dc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:33.093889 5082 raft_consensus.cc:1261] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef [term 1 FOLLOWER]: Refusing update from remote peer bfac4bf94b54447e8963ccbdcc265dc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:33.094702 5290 consensus_queue.cc:1035] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b34afabb09f24c358952a21dee40a78c" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 37643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:37:33.095046 5285 consensus_queue.cc:1035] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5367243362584706b6e016ad251540ef" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 39347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:37:33.106125 5295 mvcc.cc:205] Tried to move back new op lower bound from 7298343948661690368 to 7298343948494471168. Current Snapshot: MvccSnapshot[applied={T|T < 7298343948661690368}]
I20260618 22:37:33.119279 4804 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:33.119513 4804 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:33.123046 4804 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d15b51c0d90b44579eb7f83d991eb95a: Sending DeleteTablet for 3 replicas of tablet 5a9e801ce4564c0db1934fa36588eddf
I20260618 22:37:33.124225 5195 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a9e801ce4564c0db1934fa36588eddf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:33 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:42168
I20260618 22:37:33.124226 5062 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a9e801ce4564c0db1934fa36588eddf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:33 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:39364
I20260618 22:37:33.124668 4928 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a9e801ce4564c0db1934fa36588eddf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:33 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:48880
I20260618 22:37:33.128613 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 4865
I20260618 22:37:33.135193 5303 tablet_replica.cc:330] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: stopping tablet replica
I20260618 22:37:33.135576 5303 raft_consensus.cc:2229] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:37:33.136027 5303 raft_consensus.cc:2258] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:33.138226 5302 tablet_replica.cc:330] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: stopping tablet replica
I20260618 22:37:33.138577 5302 raft_consensus.cc:2229] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:37:33.138979 5302 raft_consensus.cc:2258] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:33.141247 5303 ts_tablet_manager.cc:1905] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:33.141247 5302 ts_tablet_manager.cc:1905] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:33.148206 5303 ts_tablet_manager.cc:1918] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:33.148206 5302 ts_tablet_manager.cc:1918] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:33.148384 5303 log.cc:1196] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/5a9e801ce4564c0db1934fa36588eddf
I20260618 22:37:33.148556 5302 log.cc:1196] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/5a9e801ce4564c0db1934fa36588eddf
I20260618 22:37:33.148914 5303 ts_tablet_manager.cc:1939] T 5a9e801ce4564c0db1934fa36588eddf P b34afabb09f24c358952a21dee40a78c: Deleting consensus metadata
I20260618 22:37:33.149179 5302 ts_tablet_manager.cc:1939] T 5a9e801ce4564c0db1934fa36588eddf P 5367243362584706b6e016ad251540ef: Deleting consensus metadata
I20260618 22:37:33.151060 4789 catalog_manager.cc:4841] TS 5367243362584706b6e016ad251540ef (127.18.150.130:39347): tablet 5a9e801ce4564c0db1934fa36588eddf (table test-table [id=971493c2a1954a8ab1a58c73997bc4e9]) successfully deleted
I20260618 22:37:33.151039 4791 catalog_manager.cc:4841] TS b34afabb09f24c358952a21dee40a78c (127.18.150.131:37643): tablet 5a9e801ce4564c0db1934fa36588eddf (table test-table [id=971493c2a1954a8ab1a58c73997bc4e9]) successfully deleted
I20260618 22:37:33.155236 5304 tablet_replica.cc:330] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: stopping tablet replica
I20260618 22:37:33.155781 5304 raft_consensus.cc:2229] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:37:33.156487 5304 raft_consensus.cc:2258] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:33.158725 5304 ts_tablet_manager.cc:1905] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 18 22:37:33 dist-test-slave-h7jz krb5kdc[4759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:37:33.164744 5304 ts_tablet_manager.cc:1918] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:33.164961 5304 log.cc:1196] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/5a9e801ce4564c0db1934fa36588eddf
I20260618 22:37:33.165514 5304 ts_tablet_manager.cc:1939] T 5a9e801ce4564c0db1934fa36588eddf P bfac4bf94b54447e8963ccbdcc265dc9: Deleting consensus metadata
I20260618 22:37:33.167600 4788 catalog_manager.cc:4841] TS bfac4bf94b54447e8963ccbdcc265dc9 (127.18.150.129:41761): tablet 5a9e801ce4564c0db1934fa36588eddf (table test-table [id=971493c2a1954a8ab1a58c73997bc4e9]) successfully deleted
I20260618 22:37:33.172549 5278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:33.129899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48902 (local address 127.18.150.129:41761)
0618 22:37:33.130093 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:33.130099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:33.154493 (+ 24394us) server_negotiation.cc:407] Connection header received
0618 22:37:33.154677 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:33.154684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:33.154777 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:33.154900 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:33.156790 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.157912 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.159948 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.160661 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.160793 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:33.166158 (+ 5365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:33.166171 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:33.166189 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:33.166232 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:33.169596 (+ 3364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:33.170425 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:33.170433 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:33.170439 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:33.170586 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:33.170977 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:33.170981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:33.170983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:33.171364 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:33.171674 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:33.171961 (+ 287us) server_negotiation.cc:299] Negotiation successful
0618 22:37:33.172201 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260618 22:37:33.173269 4988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:33.285647 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 4865
I20260618 22:37:33.300971 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 4999
Jun 18 22:37:33 dist-test-slave-h7jz krb5kdc[4759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:37:33.315052 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:33.301709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39374 (local address 127.18.150.130:39347)
0618 22:37:33.301908 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:33.301914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:33.301934 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:37:33.302066 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:33.302071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:33.302148 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:33.302243 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:33.303371 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.304573 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.305805 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.306435 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.306550 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:33.310411 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:33.310422 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:33.310434 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:33.310466 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:33.312865 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:33.313444 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:33.313451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:33.313453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:33.313559 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:33.313818 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:33.313821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:33.313823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:33.314194 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:33.314404 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:33.314620 (+ 216us) server_negotiation.cc:299] Negotiation successful
0618 22:37:33.314773 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260618 22:37:33.315701 5122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:33.402207 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 4999
I20260618 22:37:33.415061 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 5132
Jun 18 22:37:33 dist-test-slave-h7jz krb5kdc[4759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:37:33.429265 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:33.415760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42178 (local address 127.18.150.131:37643)
0618 22:37:33.415943 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:33.415948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:33.415967 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:37:33.416071 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:33.416075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:33.416149 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:33.416255 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:33.417506 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.418290 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.419489 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:33.419880 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:33.419967 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:33.424039 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:33.424054 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:33.424071 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:33.424113 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:33.426857 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:33.427487 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:33.427496 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:33.427498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:33.427650 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:33.427933 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:33.427936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:33.427938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:33.428282 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:33.428500 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:33.428732 (+ 232us) server_negotiation.cc:299] Negotiation successful
0618 22:37:33.428931 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260618 22:37:33.429919 5255 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:33.517326 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 5132
I20260618 22:37:33.530310 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 4772
I20260618 22:37:33.531359 4834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:33.641119 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 4772
2026-06-18T22:37:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4078 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-18T22:37:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:33Z Disabled control of system clock
I20260618 22:37:33.709055 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38519
--webserver_interface=127.18.150.190
--webserver_port=0
--builtin_ntp_servers=127.18.150.148:40469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:38519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:37:34.025992 5313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:34.026279 5313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:34.026399 5313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:34.033433 5313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:34.033531 5313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:34.033591 5313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:34.033666 5313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:34.033725 5313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:34.042672 5313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:40469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:38519
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:38519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:34.043865 5313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:34.045622 5313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:34.056694 5318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:34.056784 5319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:34.056761 5321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:34.057718 5313 server_base.cc:1034] running on GCE node
I20260618 22:37:34.058477 5313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:34.059793 5313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:34.060979 5313 hybrid_clock.cc:648] HybridClock initialized: now 1781822254060936 us; error 34 us; skew 500 ppm
I20260618 22:37:34.061307 5313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:34.063509 5313 webserver.cc:460] Webserver started at http://127.18.150.190:32947/ using document root <none> and password file <none>
I20260618 22:37:34.064153 5313 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:34.064236 5313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:34.064530 5313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:34.067381 5313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "a9078e7c403c41d6b763b8ab796c61d5"
format_stamp: "Formatted at 2026-06-18 22:37:34 on dist-test-slave-h7jz"
server_key: "195afe4e56d6dfbc3a32f193d1b88199"
server_key_iv: "bf220b06feec14203b2c4d8af0751a0a"
server_key_version: "encryptionkey@0"
I20260618 22:37:34.068269 5313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "a9078e7c403c41d6b763b8ab796c61d5"
format_stamp: "Formatted at 2026-06-18 22:37:34 on dist-test-slave-h7jz"
server_key: "195afe4e56d6dfbc3a32f193d1b88199"
server_key_iv: "bf220b06feec14203b2c4d8af0751a0a"
server_key_version: "encryptionkey@0"
I20260618 22:37:34.073225 5313 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260618 22:37:34.076571 5327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:34.077855 5313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:37:34.078047 5313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "a9078e7c403c41d6b763b8ab796c61d5"
format_stamp: "Formatted at 2026-06-18 22:37:34 on dist-test-slave-h7jz"
server_key: "195afe4e56d6dfbc3a32f193d1b88199"
server_key_iv: "bf220b06feec14203b2c4d8af0751a0a"
server_key_version: "encryptionkey@0"
I20260618 22:37:34.078234 5313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:34.093410 5313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:34.094309 5313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:34.094568 5313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:34.111359 5313 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:38519
I20260618 22:37:34.111382 5378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:38519 every 8 connection(s)
I20260618 22:37:34.112632 5313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:34.116230 5379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:34.119988 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 5313
I20260618 22:37:34.120173 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:34.120726 19034 external_mini_cluster.cc:1428] Setting key 3370d4647cfcf5961018dbb9fb92abb3
I20260618 22:37:34.129194 5379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5: Bootstrap starting.
I20260618 22:37:34.132786 5379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:34.133853 5379 log.cc:824] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:34.134248 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:34.122453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57954 (local address 127.18.150.190:38519)
0618 22:37:34.123421 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:34.123432 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:34.123489 (+ 57us) server_negotiation.cc:407] Connection header received
0618 22:37:34.125147 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:34.125170 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:34.125683 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:34.126100 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:34.126709 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:34.128062 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:34.129930 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:34.130666 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:34.130816 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:34.131977 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:34.131985 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:34.131997 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:34.132512 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:34.132599 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:34.132620 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:34.132706 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:34.132957 (+ 251us) server_negotiation.cc:299] Negotiation successful
0618 22:37:34.133298 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":341,"threads_started":1}
I20260618 22:37:34.136639 5379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5: No bootstrap required, opened a new log
I20260618 22:37:34.143474 5379 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9078e7c403c41d6b763b8ab796c61d5" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38519 } }
I20260618 22:37:34.143836 5379 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:34.143939 5379 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9078e7c403c41d6b763b8ab796c61d5, State: Initialized, Role: FOLLOWER
I20260618 22:37:34.144505 5379 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9078e7c403c41d6b763b8ab796c61d5" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38519 } }
I20260618 22:37:34.144717 5379 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:34.144810 5379 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:34.144934 5379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:34.146947 5379 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9078e7c403c41d6b763b8ab796c61d5" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38519 } }
I20260618 22:37:34.147357 5379 leader_election.cc:302] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9078e7c403c41d6b763b8ab796c61d5; no voters:
I20260618 22:37:34.147921 5379 leader_election.cc:288] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:34.148049 5384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:34.148762 5384 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [term 1 LEADER]: Becoming Leader. State: Replica: a9078e7c403c41d6b763b8ab796c61d5, State: Running, Role: LEADER
I20260618 22:37:34.149271 5384 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9078e7c403c41d6b763b8ab796c61d5" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38519 } }
I20260618 22:37:34.149830 5379 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:34.151877 5386 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9078e7c403c41d6b763b8ab796c61d5. Latest consensus state: current_term: 1 leader_uuid: "a9078e7c403c41d6b763b8ab796c61d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9078e7c403c41d6b763b8ab796c61d5" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38519 } } }
I20260618 22:37:34.151971 5385 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9078e7c403c41d6b763b8ab796c61d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9078e7c403c41d6b763b8ab796c61d5" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 38519 } } }
I20260618 22:37:34.152258 5386 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:34.152302 5385 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:34.155475 5390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:34.159653 5390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:34.167212 5390 catalog_manager.cc:1347] Generated new cluster ID: 1036dd45855a4dc08e7c866034f4dbb2
I20260618 22:37:34.167315 5390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:34.186556 5390 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:34.188342 5390 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:34.194751 5390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5: Generated new TSK 0
I20260618 22:37:34.195744 5390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:34.208408 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38519
--builtin_ntp_servers=127.18.150.148:40469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:37:34.520301 5403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:34.520594 5403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:34.520731 5403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:34.528044 5403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:34.528157 5403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:34.528303 5403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:37:34.537386 5403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:40469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:34.538688 5403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:34.540416 5403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:34.552942 5408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:34.552912 5409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:34.552873 5411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:34.553751 5403 server_base.cc:1034] running on GCE node
I20260618 22:37:34.554343 5403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:34.555225 5403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:34.556449 5403 hybrid_clock.cc:648] HybridClock initialized: now 1781822254556386 us; error 55 us; skew 500 ppm
I20260618 22:37:34.556761 5403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:34.558883 5403 webserver.cc:460] Webserver started at http://127.18.150.129:42101/ using document root <none> and password file <none>
I20260618 22:37:34.559553 5403 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:34.559650 5403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:34.559932 5403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:34.562731 5403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "2d9d403d5f6345949476f39b6b244810"
format_stamp: "Formatted at 2026-06-18 22:37:34 on dist-test-slave-h7jz"
server_key: "666f2c214db7461385a79e430daa3ddd"
server_key_iv: "d5aee32c14bab5119d9aa5a53a05c1fb"
server_key_version: "encryptionkey@0"
I20260618 22:37:34.563483 5403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "2d9d403d5f6345949476f39b6b244810"
format_stamp: "Formatted at 2026-06-18 22:37:34 on dist-test-slave-h7jz"
server_key: "666f2c214db7461385a79e430daa3ddd"
server_key_iv: "d5aee32c14bab5119d9aa5a53a05c1fb"
server_key_version: "encryptionkey@0"
I20260618 22:37:34.568480 5403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:37:34.571801 5417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:34.573071 5403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:37:34.573233 5403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "2d9d403d5f6345949476f39b6b244810"
format_stamp: "Formatted at 2026-06-18 22:37:34 on dist-test-slave-h7jz"
server_key: "666f2c214db7461385a79e430daa3ddd"
server_key_iv: "d5aee32c14bab5119d9aa5a53a05c1fb"
server_key_version: "encryptionkey@0"
I20260618 22:37:34.573421 5403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:34.593593 5403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:34.594441 5403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:34.594719 5403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:34.595630 5403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:34.597014 5403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:34.597106 5403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:34.597213 5403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:34.597280 5403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:34.626108 5403 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:35665
I20260618 22:37:34.626173 5529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:35665 every 8 connection(s)
I20260618 22:37:34.627396 5403 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:37:34.629443 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 5403
I20260618 22:37:34.629601 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:37:34.630152 19034 external_mini_cluster.cc:1428] Setting key 4c45060b679d6c39af8db469278017f7
I20260618 22:37:34.633184 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38519
--builtin_ntp_servers=127.18.150.148:40469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:37:34.640965 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:34.630013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:60641 (local address 127.18.150.190:38519)
0618 22:37:34.630222 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:34.630228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:34.631626 (+ 1398us) server_negotiation.cc:407] Connection header received
0618 22:37:34.634039 (+ 2413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:34.634042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:34.634103 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:34.634178 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:34.635576 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:34.636427 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:34.638469 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:34.638877 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:34.638973 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:34.640345 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:34.640350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:34.640353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:34.640370 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:34.640401 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:34.640412 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:34.640477 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:34.640665 (+ 188us) server_negotiation.cc:299] Negotiation successful
0618 22:37:34.640745 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:37:34.642305 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:34.630526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38519 (local address 127.18.150.129:60641)
0618 22:37:34.631487 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:34.631522 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:34.633694 (+ 2172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:34.634445 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:34.634457 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:34.634580 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:34.635340 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:34.635363 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:34.636582 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:34.636587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:34.638335 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:34.638350 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:34.639079 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:34.639086 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:34.639354 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:34.640028 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:37:34.640031 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:34.640080 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:37:34.640179 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:34.640490 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:34.640498 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:34.640505 (+ 7us) client_negotiation.cc:769] Sending connection context
0618 22:37:34.640662 (+ 157us) client_negotiation.cc:240] Negotiation successful
0618 22:37:34.641047 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":312,"threads_started":1}
I20260618 22:37:34.643826 5530 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38519
I20260618 22:37:34.644114 5530 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:34.644845 5530 heartbeater.cc:511] Master 127.18.150.190:38519 requested a full tablet report, sending...
I20260618 22:37:34.646791 5344 ts_manager.cc:194] Registered new tserver with Master: 2d9d403d5f6345949476f39b6b244810 (127.18.150.129:35665)
I20260618 22:37:34.648211 5344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.129:60641
W20260618 22:37:34.945313 5533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:34.945608 5533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:34.945729 5533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:34.953029 5533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:34.953141 5533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:34.953284 5533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:37:34.962568 5533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:40469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:34.963829 5533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:34.965504 5533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:34.977581 5541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:34.977612 5539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:34.977803 5538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:34.978160 5533 server_base.cc:1034] running on GCE node
I20260618 22:37:34.978796 5533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:34.979653 5533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:34.980861 5533 hybrid_clock.cc:648] HybridClock initialized: now 1781822254980809 us; error 36 us; skew 500 ppm
I20260618 22:37:34.981150 5533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:34.983219 5533 webserver.cc:460] Webserver started at http://127.18.150.130:34861/ using document root <none> and password file <none>
I20260618 22:37:34.983873 5533 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:34.983975 5533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:34.984251 5533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:34.986995 5533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "f3242faaeedb423baba9298dc83dcb5c"
format_stamp: "Formatted at 2026-06-18 22:37:34 on dist-test-slave-h7jz"
server_key: "625616ad61f7fe5f3ca39abfc05b6fd3"
server_key_iv: "f1a2e83292ac190fb4eebc2959cb901d"
server_key_version: "encryptionkey@0"
I20260618 22:37:34.987710 5533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "f3242faaeedb423baba9298dc83dcb5c"
format_stamp: "Formatted at 2026-06-18 22:37:34 on dist-test-slave-h7jz"
server_key: "625616ad61f7fe5f3ca39abfc05b6fd3"
server_key_iv: "f1a2e83292ac190fb4eebc2959cb901d"
server_key_version: "encryptionkey@0"
I20260618 22:37:34.992476 5533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260618 22:37:34.995699 5547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:34.996971 5533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260618 22:37:34.997112 5533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "f3242faaeedb423baba9298dc83dcb5c"
format_stamp: "Formatted at 2026-06-18 22:37:34 on dist-test-slave-h7jz"
server_key: "625616ad61f7fe5f3ca39abfc05b6fd3"
server_key_iv: "f1a2e83292ac190fb4eebc2959cb901d"
server_key_version: "encryptionkey@0"
I20260618 22:37:34.997308 5533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:35.023098 5533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:35.023957 5533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:35.024233 5533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:35.025261 5533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:35.026721 5533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:35.026810 5533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:35.026916 5533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:35.026984 5533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:35.054867 5533 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:42263
I20260618 22:37:35.054929 5659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:42263 every 8 connection(s)
I20260618 22:37:35.056149 5533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:37:35.058326 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 5533
I20260618 22:37:35.058503 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:37:35.059041 19034 external_mini_cluster.cc:1428] Setting key 487c3c874bddd4751689b095ea7145f9
I20260618 22:37:35.061874 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38519
--builtin_ntp_servers=127.18.150.148:40469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:37:35.068544 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.058576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:58729 (local address 127.18.150.190:38519)
0618 22:37:35.058746 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:35.058750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:35.060437 (+ 1687us) server_negotiation.cc:407] Connection header received
0618 22:37:35.062417 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:35.062423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:35.062499 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:35.062598 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:35.063545 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.064461 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.065983 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.066561 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.066685 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.067836 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:35.067842 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:35.067847 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:35.067867 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:35.067904 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:35.067918 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:35.067995 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:35.068148 (+ 153us) server_negotiation.cc:299] Negotiation successful
0618 22:37:35.068238 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:37:35.069402 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.059243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38519 (local address 127.18.150.130:58729)
0618 22:37:35.060276 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:35.060324 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:35.062172 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:35.062774 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:35.062782 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:35.062873 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:35.063401 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.063416 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.064578 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.064582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.065833 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.065846 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.066751 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.066756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.066936 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.067551 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:37:35.067553 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:35.067589 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:37:35.067662 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:35.067991 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:35.067997 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:35.068002 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:37:35.068123 (+ 121us) client_negotiation.cc:240] Negotiation successful
0618 22:37:35.068389 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":290,"threads_started":1}
I20260618 22:37:35.070772 5660 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38519
I20260618 22:37:35.071067 5660 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:35.071851 5660 heartbeater.cc:511] Master 127.18.150.190:38519 requested a full tablet report, sending...
I20260618 22:37:35.073230 5344 ts_manager.cc:194] Registered new tserver with Master: f3242faaeedb423baba9298dc83dcb5c (127.18.150.130:42263)
I20260618 22:37:35.074070 5344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.130:58729
W20260618 22:37:35.371407 5663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:35.371672 5663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:35.371799 5663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:35.378995 5663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:35.379122 5663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:35.379258 5663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:37:35.388315 5663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:40469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:38519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:35.389474 5663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:35.391149 5663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:35.403373 5668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:35.403348 5669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:35.403555 5671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:35.404063 5663 server_base.cc:1034] running on GCE node
I20260618 22:37:35.404660 5663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:35.405547 5663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:35.406760 5663 hybrid_clock.cc:648] HybridClock initialized: now 1781822255406714 us; error 38 us; skew 500 ppm
I20260618 22:37:35.407050 5663 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:35.409080 5663 webserver.cc:460] Webserver started at http://127.18.150.131:44533/ using document root <none> and password file <none>
I20260618 22:37:35.409708 5663 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:35.409803 5663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:35.410117 5663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:35.412760 5663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "d30d498ff94448fea228e0e6a4ebdf36"
format_stamp: "Formatted at 2026-06-18 22:37:35 on dist-test-slave-h7jz"
server_key: "9158fd5300b94f3dc0e49b879f823f1c"
server_key_iv: "5c7af77b4e3126ed4dcebf37c6deba5d"
server_key_version: "encryptionkey@0"
I20260618 22:37:35.413468 5663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "d30d498ff94448fea228e0e6a4ebdf36"
format_stamp: "Formatted at 2026-06-18 22:37:35 on dist-test-slave-h7jz"
server_key: "9158fd5300b94f3dc0e49b879f823f1c"
server_key_iv: "5c7af77b4e3126ed4dcebf37c6deba5d"
server_key_version: "encryptionkey@0"
I20260618 22:37:35.418004 5663 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260618 22:37:35.421041 5677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:35.422250 5663 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:37:35.422396 5663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "d30d498ff94448fea228e0e6a4ebdf36"
format_stamp: "Formatted at 2026-06-18 22:37:35 on dist-test-slave-h7jz"
server_key: "9158fd5300b94f3dc0e49b879f823f1c"
server_key_iv: "5c7af77b4e3126ed4dcebf37c6deba5d"
server_key_version: "encryptionkey@0"
I20260618 22:37:35.422570 5663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:35.436714 5663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:35.437454 5663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:35.437718 5663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:35.438632 5663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:35.439929 5663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:35.440021 5663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:35.440131 5663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:35.440200 5663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:35.468139 5663 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:38393
I20260618 22:37:35.468161 5789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:38393 every 8 connection(s)
I20260618 22:37:35.469377 5663 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:37:35.476135 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 5663
I20260618 22:37:35.476301 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:37:35.476877 19034 external_mini_cluster.cc:1428] Setting key bb72d7792a936517eaceb1adb5a81536
I20260618 22:37:35.481319 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.471776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:52599 (local address 127.18.150.190:38519)
0618 22:37:35.471958 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:35.471962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:35.473315 (+ 1353us) server_negotiation.cc:407] Connection header received
0618 22:37:35.475033 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:35.475039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:35.475125 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:35.475222 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:35.476331 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.477349 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.478921 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.479325 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.479430 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.480581 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:35.480585 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:35.480589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:35.480605 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:35.480634 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:35.480645 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:35.480717 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:35.480918 (+ 201us) server_negotiation.cc:299] Negotiation successful
0618 22:37:35.481011 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:37:35.482352 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.472275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:38519 (local address 127.18.150.131:52599)
0618 22:37:35.473138 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:35.473178 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:35.474768 (+ 1590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:35.475474 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:35.475482 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:35.475570 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:35.476143 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.476160 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.477495 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.477499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.478775 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.478787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.479525 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.479533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.479741 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.480308 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:37:35.480311 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:35.480349 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:37:35.480421 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:35.480738 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:35.480748 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:35.480754 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:37:35.480896 (+ 142us) client_negotiation.cc:240] Negotiation successful
0618 22:37:35.481282 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":262,"threads_started":1}
I20260618 22:37:35.483528 5790 heartbeater.cc:347] Connected to a master server at 127.18.150.190:38519
I20260618 22:37:35.483774 5790 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:35.484429 5790 heartbeater.cc:511] Master 127.18.150.190:38519 requested a full tablet report, sending...
I20260618 22:37:35.485635 5344 ts_manager.cc:194] Registered new tserver with Master: d30d498ff94448fea228e0e6a4ebdf36 (127.18.150.131:38393)
I20260618 22:37:35.486413 5344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.131:52599
I20260618 22:37:35.491564 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:37:35.501272 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.495483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57966 (local address 127.18.150.190:38519)
0618 22:37:35.495637 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:35.495641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:35.496063 (+ 422us) server_negotiation.cc:407] Connection header received
0618 22:37:35.496221 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:35.496224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:35.496315 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:35.496396 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:35.496860 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.497579 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.499021 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.499397 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.499500 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.500564 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:35.500569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:35.500572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:35.500588 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:35.500617 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:35.500628 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:35.500744 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:35.500971 (+ 227us) server_negotiation.cc:299] Negotiation successful
0618 22:37:35.501025 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:37:35.504915 5344 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:37:35.511482 5344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:37:35.536636 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.522530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:38393 (local address 127.0.0.1:38264)
0618 22:37:35.523511 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:35.523549 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:35.523708 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:35.525489 (+ 1781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:35.525498 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:35.525533 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:35.526094 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.526109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.529620 (+ 3511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.529626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.533337 (+ 3711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.533356 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.535869 (+ 2513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.535877 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.536066 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.536105 (+ 39us) client_negotiation.cc:769] Sending connection context
0618 22:37:35.536196 (+ 91us) client_negotiation.cc:240] Negotiation successful
0618 22:37:35.536339 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":788,"threads_started":1}
I20260618 22:37:35.537351 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.522890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.18.150.131:38393)
0618 22:37:35.524700 (+ 1810us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:35.524720 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:35.524753 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:37:35.524873 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:35.524882 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:35.525139 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:35.527500 (+ 2361us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:35.527799 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.529454 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.533621 (+ 4167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.535642 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.536540 (+ 898us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.536764 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:35.536834 (+ 70us) server_negotiation.cc:299] Negotiation successful
0618 22:37:35.536994 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1661,"thread_start_us":1515,"threads_started":1}
I20260618 22:37:35.541121 5725 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb (DEFAULT_TABLE table=test-table [id=bc116c5b77ef455bb25827504103b2cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:35.542273 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.524154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:42263 (local address 127.0.0.1:51200)
0618 22:37:35.534892 (+ 10738us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:35.534908 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:35.535019 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:35.535680 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:35.535685 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:35.535727 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:35.536305 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.536315 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.538022 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.538030 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.540192 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.540213 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.541448 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.541455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.541620 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.541651 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:37:35.541745 (+ 94us) client_negotiation.cc:240] Negotiation successful
0618 22:37:35.541868 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10618,"mutex_wait_us":83,"spinlock_wait_cycles":13952,"thread_start_us":9884,"threads_started":1}
I20260618 22:37:35.542307 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.524636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51200 (local address 127.18.150.130:42263)
0618 22:37:35.534580 (+ 9944us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:35.534589 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:35.535134 (+ 545us) server_negotiation.cc:407] Connection header received
0618 22:37:35.535247 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:35.535255 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:35.535486 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:35.535676 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:35.536471 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.537851 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.540414 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.541278 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.541454 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.541651 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:35.541788 (+ 137us) server_negotiation.cc:299] Negotiation successful
0618 22:37:35.541973 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9771,"thread_start_us":9656,"threads_started":1}
I20260618 22:37:35.542963 5725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:35.542987 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.523111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:35665 (local address 127.0.0.1:49680)
0618 22:37:35.534283 (+ 11172us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:35.534303 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:35.534883 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:35.535692 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:35.535695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:35.535724 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:35.536227 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.536242 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.537907 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.537912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.540194 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.540207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.542236 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.542244 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.542432 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.542462 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:37:35.542540 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:37:35.542656 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11043,"mutex_wait_us":46,"spinlock_wait_cycles":768,"thread_start_us":11514,"threads_started":1}
I20260618 22:37:35.543232 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.523519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49680 (local address 127.18.150.129:35665)
0618 22:37:35.524215 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:35.524233 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:35.534412 (+ 10179us) server_negotiation.cc:407] Connection header received
0618 22:37:35.535096 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:35.535105 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:35.535363 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:35.535489 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:35.536448 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.537790 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.540394 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.541711 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.541867 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.542088 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:35.542733 (+ 645us) server_negotiation.cc:299] Negotiation successful
0618 22:37:35.542903 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":729,"threads_started":1}
I20260618 22:37:35.545673 5595 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb (DEFAULT_TABLE table=test-table [id=bc116c5b77ef455bb25827504103b2cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:35.546885 5595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:35.546612 5465 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb (DEFAULT_TABLE table=test-table [id=bc116c5b77ef455bb25827504103b2cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:35.547847 5465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:35.557245 5808 tablet_bootstrap.cc:492] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: Bootstrap starting.
I20260618 22:37:35.558995 5809 tablet_bootstrap.cc:492] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c: Bootstrap starting.
I20260618 22:37:35.560588 5808 tablet_bootstrap.cc:654] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:35.561803 5808 log.cc:824] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:35.562563 5809 tablet_bootstrap.cc:654] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:35.563787 5809 log.cc:824] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:35.564559 5808 tablet_bootstrap.cc:492] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: No bootstrap required, opened a new log
I20260618 22:37:35.564793 5808 ts_tablet_manager.cc:1397] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260618 22:37:35.566052 5809 tablet_bootstrap.cc:492] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c: No bootstrap required, opened a new log
I20260618 22:37:35.566272 5809 ts_tablet_manager.cc:1397] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260618 22:37:35.570097 5810 tablet_bootstrap.cc:492] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: Bootstrap starting.
I20260618 22:37:35.571938 5808 raft_consensus.cc:348] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30d498ff94448fea228e0e6a4ebdf36" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38393 } } peers { permanent_uuid: "f3242faaeedb423baba9298dc83dcb5c" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42263 } } peers { permanent_uuid: "2d9d403d5f6345949476f39b6b244810" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35665 } }
I20260618 22:37:35.572477 5808 raft_consensus.cc:374] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:35.572614 5808 raft_consensus.cc:729] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d30d498ff94448fea228e0e6a4ebdf36, State: Initialized, Role: FOLLOWER
I20260618 22:37:35.573297 5809 raft_consensus.cc:348] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30d498ff94448fea228e0e6a4ebdf36" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38393 } } peers { permanent_uuid: "f3242faaeedb423baba9298dc83dcb5c" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42263 } } peers { permanent_uuid: "2d9d403d5f6345949476f39b6b244810" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35665 } }
I20260618 22:37:35.573573 5810 tablet_bootstrap.cc:654] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:35.573402 5808 consensus_queue.cc:260] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30d498ff94448fea228e0e6a4ebdf36" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38393 } } peers { permanent_uuid: "f3242faaeedb423baba9298dc83dcb5c" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42263 } } peers { permanent_uuid: "2d9d403d5f6345949476f39b6b244810" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35665 } }
I20260618 22:37:35.573601 5809 raft_consensus.cc:374] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:35.573679 5809 raft_consensus.cc:729] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3242faaeedb423baba9298dc83dcb5c, State: Initialized, Role: FOLLOWER
I20260618 22:37:35.574308 5809 consensus_queue.cc:260] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30d498ff94448fea228e0e6a4ebdf36" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38393 } } peers { permanent_uuid: "f3242faaeedb423baba9298dc83dcb5c" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42263 } } peers { permanent_uuid: "2d9d403d5f6345949476f39b6b244810" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35665 } }
I20260618 22:37:35.574465 5790 heartbeater.cc:503] Master 127.18.150.190:38519 was elected leader, sending a full tablet report...
I20260618 22:37:35.574822 5810 log.cc:824] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:35.575249 5660 heartbeater.cc:503] Master 127.18.150.190:38519 was elected leader, sending a full tablet report...
I20260618 22:37:35.575850 5809 ts_tablet_manager.cc:1428] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260618 22:37:35.577039 5808 ts_tablet_manager.cc:1428] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: Time spent starting tablet: real 0.012s user 0.000s sys 0.011s
I20260618 22:37:35.579491 5810 tablet_bootstrap.cc:492] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: No bootstrap required, opened a new log
I20260618 22:37:35.579802 5810 ts_tablet_manager.cc:1397] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.009s
I20260618 22:37:35.587040 5810 raft_consensus.cc:348] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30d498ff94448fea228e0e6a4ebdf36" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38393 } } peers { permanent_uuid: "f3242faaeedb423baba9298dc83dcb5c" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42263 } } peers { permanent_uuid: "2d9d403d5f6345949476f39b6b244810" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35665 } }
I20260618 22:37:35.587375 5810 raft_consensus.cc:374] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:35.587464 5810 raft_consensus.cc:729] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d9d403d5f6345949476f39b6b244810, State: Initialized, Role: FOLLOWER
I20260618 22:37:35.587996 5810 consensus_queue.cc:260] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30d498ff94448fea228e0e6a4ebdf36" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38393 } } peers { permanent_uuid: "f3242faaeedb423baba9298dc83dcb5c" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42263 } } peers { permanent_uuid: "2d9d403d5f6345949476f39b6b244810" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35665 } }
I20260618 22:37:35.588871 5530 heartbeater.cc:503] Master 127.18.150.190:38519 was elected leader, sending a full tablet report...
I20260618 22:37:35.589306 5810 ts_tablet_manager.cc:1428] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
W20260618 22:37:35.629418 5531 tablet.cc:2369] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:37:35.660742 5813 raft_consensus.cc:482] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:37:35.661027 5813 raft_consensus.cc:504] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30d498ff94448fea228e0e6a4ebdf36" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38393 } } peers { permanent_uuid: "f3242faaeedb423baba9298dc83dcb5c" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42263 } } peers { permanent_uuid: "2d9d403d5f6345949476f39b6b244810" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35665 } }
I20260618 22:37:35.662758 5813 leader_election.cc:288] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3242faaeedb423baba9298dc83dcb5c (127.18.150.130:42263), 2d9d403d5f6345949476f39b6b244810 (127.18.150.129:35665)
I20260618 22:37:35.668218 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.663036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:35665 (local address 127.18.150.131:33539)
0618 22:37:35.663403 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:35.663422 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:35.663551 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:35.663933 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:35.663939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:35.663966 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:35.664110 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.664118 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.665069 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.665072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.666491 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.666503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.667518 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.667523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.667675 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.667698 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:37:35.667776 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:37:35.667907 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":16}
I20260618 22:37:35.668210 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.663184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:33539 (local address 127.18.150.129:35665)
0618 22:37:35.663590 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:35.663596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:35.663619 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:37:35.663714 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:35.663719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:35.663798 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:35.663872 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:35.664251 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.664930 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.666645 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.667376 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.667512 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.667581 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:35.667819 (+ 238us) server_negotiation.cc:299] Negotiation successful
0618 22:37:35.667941 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261}
I20260618 22:37:35.668502 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.663208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:48081 (local address 127.18.150.130:42263)
0618 22:37:35.663403 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:35.663409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:35.663897 (+ 488us) server_negotiation.cc:407] Connection header received
0618 22:37:35.664062 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:35.664066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:35.664121 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:35.664194 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:35.664674 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.665342 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.667014 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.667717 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.667872 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.667936 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:35.668161 (+ 225us) server_negotiation.cc:299] Negotiation successful
0618 22:37:35.668276 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:37:35.668567 5817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.663125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:42263 (local address 127.18.150.131:48081)
0618 22:37:35.663798 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:35.663813 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:35.663937 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:35.664217 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:35.664221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:35.664244 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:35.664531 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.664539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.665483 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.665487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.666854 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:35.666865 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.667864 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.667870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:35.668020 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.668046 (+ 26us) client_negotiation.cc:769] Sending connection context
0618 22:37:35.668117 (+ 71us) client_negotiation.cc:240] Negotiation successful
0618 22:37:35.668229 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":5,"thread_start_us":350,"threads_started":1}
I20260618 22:37:35.669209 5485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a8fca7bfb4429a9f70a05d3d02eeb" candidate_uuid: "d30d498ff94448fea228e0e6a4ebdf36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d9d403d5f6345949476f39b6b244810" is_pre_election: true
I20260618 22:37:35.669368 5615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a8fca7bfb4429a9f70a05d3d02eeb" candidate_uuid: "d30d498ff94448fea228e0e6a4ebdf36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3242faaeedb423baba9298dc83dcb5c" is_pre_election: true
I20260618 22:37:35.669691 5485 raft_consensus.cc:2454] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d30d498ff94448fea228e0e6a4ebdf36 in term 0.
I20260618 22:37:35.669767 5615 raft_consensus.cc:2454] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d30d498ff94448fea228e0e6a4ebdf36 in term 0.
I20260618 22:37:35.670315 5678 leader_election.cc:302] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d9d403d5f6345949476f39b6b244810, d30d498ff94448fea228e0e6a4ebdf36; no voters:
I20260618 22:37:35.670671 5813 raft_consensus.cc:2781] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:37:35.670819 5813 raft_consensus.cc:482] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:37:35.670919 5813 raft_consensus.cc:3037] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:35.672972 5813 raft_consensus.cc:504] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30d498ff94448fea228e0e6a4ebdf36" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38393 } } peers { permanent_uuid: "f3242faaeedb423baba9298dc83dcb5c" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42263 } } peers { permanent_uuid: "2d9d403d5f6345949476f39b6b244810" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35665 } }
I20260618 22:37:35.673480 5813 leader_election.cc:288] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [CANDIDATE]: Term 1 election: Requested vote from peers f3242faaeedb423baba9298dc83dcb5c (127.18.150.130:42263), 2d9d403d5f6345949476f39b6b244810 (127.18.150.129:35665)
I20260618 22:37:35.673902 5615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a8fca7bfb4429a9f70a05d3d02eeb" candidate_uuid: "d30d498ff94448fea228e0e6a4ebdf36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3242faaeedb423baba9298dc83dcb5c"
I20260618 22:37:35.673977 5485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a8fca7bfb4429a9f70a05d3d02eeb" candidate_uuid: "d30d498ff94448fea228e0e6a4ebdf36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d9d403d5f6345949476f39b6b244810"
I20260618 22:37:35.674151 5485 raft_consensus.cc:3037] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:35.674145 5615 raft_consensus.cc:3037] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:35.677218 5615 raft_consensus.cc:2454] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d30d498ff94448fea228e0e6a4ebdf36 in term 1.
I20260618 22:37:35.677217 5485 raft_consensus.cc:2454] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d30d498ff94448fea228e0e6a4ebdf36 in term 1.
I20260618 22:37:35.677599 5680 leader_election.cc:302] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d30d498ff94448fea228e0e6a4ebdf36, f3242faaeedb423baba9298dc83dcb5c; no voters:
I20260618 22:37:35.677906 5813 raft_consensus.cc:2781] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:35.678352 5813 raft_consensus.cc:686] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 1 LEADER]: Becoming Leader. State: Replica: d30d498ff94448fea228e0e6a4ebdf36, State: Running, Role: LEADER
I20260618 22:37:35.678812 5813 consensus_queue.cc:237] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30d498ff94448fea228e0e6a4ebdf36" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38393 } } peers { permanent_uuid: "f3242faaeedb423baba9298dc83dcb5c" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42263 } } peers { permanent_uuid: "2d9d403d5f6345949476f39b6b244810" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35665 } }
I20260618 22:37:35.683336 5344 catalog_manager.cc:5495] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 reported cstate change: term changed from 0 to 1, leader changed from <none> to d30d498ff94448fea228e0e6a4ebdf36 (127.18.150.131). New cstate: current_term: 1 leader_uuid: "d30d498ff94448fea228e0e6a4ebdf36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d30d498ff94448fea228e0e6a4ebdf36" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 38393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3242faaeedb423baba9298dc83dcb5c" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d9d403d5f6345949476f39b6b244810" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35665 } health_report { overall_health: UNKNOWN } } }
W20260618 22:37:35.720892 5791 tablet.cc:2369] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:37:35.723644 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.718371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.18.150.131:38393)
0618 22:37:35.718565 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:35.718571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:35.718590 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:37:35.718764 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:35.718768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:35.718846 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:35.718969 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:37:35.719336 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.720022 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.721666 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.722138 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.722242 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.722669 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:37:35.723154 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:37:35.723286 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:35.723358 (+ 72us) server_negotiation.cc:299] Negotiation successful
0618 22:37:35.723393 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:37:35.738742 5615 raft_consensus.cc:1261] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c [term 1 FOLLOWER]: Refusing update from remote peer d30d498ff94448fea228e0e6a4ebdf36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:35.738853 5485 raft_consensus.cc:1261] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810 [term 1 FOLLOWER]: Refusing update from remote peer d30d498ff94448fea228e0e6a4ebdf36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:35.739666 5818 consensus_queue.cc:1035] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3242faaeedb423baba9298dc83dcb5c" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:37:35.740031 5813 consensus_queue.cc:1035] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d9d403d5f6345949476f39b6b244810" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 35665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:37:35.751327 5822 mvcc.cc:205] Tried to move back new op lower bound from 7298343959495135232 to 7298343959264256000. Current Snapshot: MvccSnapshot[applied={T|T < 7298343959495135232}]
I20260618 22:37:35.764163 5343 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:35.764460 5343 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:35.768110 5343 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9078e7c403c41d6b763b8ab796c61d5: Sending DeleteTablet for 3 replicas of tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb
I20260618 22:37:35.769455 5595 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:35 UTC) from {username='slave'} at 127.0.0.1:51200
I20260618 22:37:35.769459 5725 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:35 UTC) from {username='slave'} at 127.0.0.1:38264
I20260618 22:37:35.769718 5465 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:35 UTC) from {username='slave'} at 127.0.0.1:49680
I20260618 22:37:35.771528 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 5403
I20260618 22:37:35.777642 5833 tablet_replica.cc:330] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: stopping tablet replica
I20260618 22:37:35.778138 5833 raft_consensus.cc:2229] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:37:35.778620 5831 tablet_replica.cc:330] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: stopping tablet replica
I20260618 22:37:35.778683 5833 raft_consensus.cc:2258] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:35.779014 5831 raft_consensus.cc:2229] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:37:35.779099 5832 tablet_replica.cc:330] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c: stopping tablet replica
I20260618 22:37:35.779690 5831 raft_consensus.cc:2258] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:35.779722 5832 raft_consensus.cc:2229] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:37:35.780524 5832 raft_consensus.cc:2258] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:35.781721 5833 ts_tablet_manager.cc:1905] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:35.781711 5831 ts_tablet_manager.cc:1905] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:35.782861 5832 ts_tablet_manager.cc:1905] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:35.788756 5833 ts_tablet_manager.cc:1918] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:35.788857 5831 ts_tablet_manager.cc:1918] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:35.788945 5833 log.cc:1196] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/1e8a8fca7bfb4429a9f70a05d3d02eeb
I20260618 22:37:35.789050 5831 log.cc:1196] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/1e8a8fca7bfb4429a9f70a05d3d02eeb
I20260618 22:37:35.789510 5833 ts_tablet_manager.cc:1939] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P 2d9d403d5f6345949476f39b6b244810: Deleting consensus metadata
I20260618 22:37:35.789518 5832 ts_tablet_manager.cc:1918] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:35.789616 5831 ts_tablet_manager.cc:1939] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P d30d498ff94448fea228e0e6a4ebdf36: Deleting consensus metadata
I20260618 22:37:35.789674 5832 log.cc:1196] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/1e8a8fca7bfb4429a9f70a05d3d02eeb
I20260618 22:37:35.790258 5832 ts_tablet_manager.cc:1939] T 1e8a8fca7bfb4429a9f70a05d3d02eeb P f3242faaeedb423baba9298dc83dcb5c: Deleting consensus metadata
I20260618 22:37:35.791430 5330 catalog_manager.cc:4841] TS d30d498ff94448fea228e0e6a4ebdf36 (127.18.150.131:38393): tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb (table test-table [id=bc116c5b77ef455bb25827504103b2cc]) successfully deleted
I20260618 22:37:35.791785 5330 catalog_manager.cc:4841] TS f3242faaeedb423baba9298dc83dcb5c (127.18.150.130:42263): tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb (table test-table [id=bc116c5b77ef455bb25827504103b2cc]) successfully deleted
I20260618 22:37:35.791893 5328 catalog_manager.cc:4841] TS 2d9d403d5f6345949476f39b6b244810 (127.18.150.129:35665): tablet 1e8a8fca7bfb4429a9f70a05d3d02eeb (table test-table [id=bc116c5b77ef455bb25827504103b2cc]) successfully deleted
I20260618 22:37:35.793526 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.772922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49696 (local address 127.18.150.129:35665)
0618 22:37:35.773110 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:35.773117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:35.777821 (+ 4704us) server_negotiation.cc:407] Connection header received
0618 22:37:35.778022 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:35.778027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:35.778155 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:35.778262 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:35.778934 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.780035 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.782267 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.782928 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.783023 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.791946 (+ 8923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:35.791957 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:35.791973 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:35.792582 (+ 609us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:35.792631 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:35.792651 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:35.792717 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:35.792971 (+ 254us) server_negotiation.cc:299] Negotiation successful
0618 22:37:35.793092 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:37:35.794220 5525 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:35.881084 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 5403
I20260618 22:37:35.895329 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 5533
I20260618 22:37:35.902563 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:35.896054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51208 (local address 127.18.150.130:42263)
0618 22:37:35.896239 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:35.896245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:35.896266 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:37:35.896385 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:35.896390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:35.896505 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:35.896615 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:35.896945 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.898033 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.899194 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:35.899806 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:35.899927 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:35.900891 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:35.900904 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:35.900923 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:35.901729 (+ 806us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:35.901803 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:35.901828 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:35.901961 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:35.902133 (+ 172us) server_negotiation.cc:299] Negotiation successful
0618 22:37:35.902238 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:37:35.903162 5655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:35.999042 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 5533
I20260618 22:37:36.017352 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 5663
I20260618 22:37:36.025013 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:36.018114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.18.150.131:38393)
0618 22:37:36.018289 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:36.018293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:36.018381 (+ 88us) server_negotiation.cc:407] Connection header received
0618 22:37:36.018561 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:36.018566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:36.018672 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:36.018769 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:36.019231 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:36.020016 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:36.021861 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:36.022413 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:36.022508 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:36.023662 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:36.023671 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:36.023688 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:36.024267 (+ 579us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:36.024314 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:36.024340 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:36.024415 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:36.024579 (+ 164us) server_negotiation.cc:299] Negotiation successful
0618 22:37:36.024657 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:37:36.025760 5785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:36.144052 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 5663
I20260618 22:37:36.159299 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 5313
I20260618 22:37:36.160310 5374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:36.265909 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 5313
2026-06-18T22:37:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2623 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:36 dist-test-slave-h7jz krb5kdc[5841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:36 dist-test-slave-h7jz krb5kdc[5841](info): set up 2 sockets
Jun 18 22:37:36 dist-test-slave-h7jz krb5kdc[5841](info): commencing operation
krb5kdc: starting...
W20260618 22:37:37.270524 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.969s user 0.009s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:37 dist-test-slave-h7jz krb5kdc[5841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:37:37.509377 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:41725
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:41725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:37.824334 5854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:37.824625 5854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:37.824749 5854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:37.832154 5854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:37.832262 5854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:37.832320 5854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:37.832396 5854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:37.832443 5854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:37.842348 5854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:41725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:41725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:37.843636 5854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:37.844930 5854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:37.855790 5860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:37.855852 5862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:37.855845 5859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:37.856675 5854 server_base.cc:1034] running on GCE node
I20260618 22:37:37.857405 5854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:37.858768 5854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:37.859958 5854 hybrid_clock.cc:648] HybridClock initialized: now 1781822257859907 us; error 38 us; skew 500 ppm
I20260618 22:37:37.860272 5854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:37 dist-test-slave-h7jz krb5kdc[5841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822257, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:37.866752 5854 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:37:37.868201 5854 webserver.cc:460] Webserver started at http://127.18.150.190:36625/ using document root <none> and password file <none>
I20260618 22:37:37.868870 5854 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:37.868981 5854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:37.869249 5854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:37.872130 5854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "b2e8b54d61544f70a0486eec4e5661ff"
format_stamp: "Formatted at 2026-06-18 22:37:37 on dist-test-slave-h7jz"
server_key: "2f4cfff8d973deb22171997c9ca8b47e"
server_key_iv: "7259d56e2701d14ac70fd602a36529aa"
server_key_version: "encryptionkey@0"
I20260618 22:37:37.872831 5854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "b2e8b54d61544f70a0486eec4e5661ff"
format_stamp: "Formatted at 2026-06-18 22:37:37 on dist-test-slave-h7jz"
server_key: "2f4cfff8d973deb22171997c9ca8b47e"
server_key_iv: "7259d56e2701d14ac70fd602a36529aa"
server_key_version: "encryptionkey@0"
I20260618 22:37:37.877729 5854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260618 22:37:37.881011 5869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:37.882302 5854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:37:37.882457 5854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "b2e8b54d61544f70a0486eec4e5661ff"
format_stamp: "Formatted at 2026-06-18 22:37:37 on dist-test-slave-h7jz"
server_key: "2f4cfff8d973deb22171997c9ca8b47e"
server_key_iv: "7259d56e2701d14ac70fd602a36529aa"
server_key_version: "encryptionkey@0"
I20260618 22:37:37.882630 5854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:37.909304 5854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:37.913110 5854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:37.913365 5854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:37.930166 5854 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:41725
I20260618 22:37:37.930168 5920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:41725 every 8 connection(s)
I20260618 22:37:37.931559 5854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:37.935182 5921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:37.940287 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 5854
I20260618 22:37:37.940474 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:37.940974 19034 external_mini_cluster.cc:1428] Setting key 0566d5d2f359f4980b5bb356b6829e54
I20260618 22:37:37.946010 5921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff: Bootstrap starting.
I20260618 22:37:37.949460 5921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:37.950719 5921 log.cc:824] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:37.954102 5921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff: No bootstrap required, opened a new log
Jun 18 22:37:37 dist-test-slave-h7jz krb5kdc[5841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:37.962716 5921 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e8b54d61544f70a0486eec4e5661ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41725 } }
I20260618 22:37:37.963141 5921 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:37.963250 5921 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e8b54d61544f70a0486eec4e5661ff, State: Initialized, Role: FOLLOWER
I20260618 22:37:37.963809 5921 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e8b54d61544f70a0486eec4e5661ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41725 } }
I20260618 22:37:37.964021 5921 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:37.964155 5921 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:37.964185 5924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:37.942826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57996 (local address 127.18.150.190:41725)
0618 22:37:37.943593 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:37.943606 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:37.943644 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:37:37.945332 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:37.945355 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:37.945723 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:37.946071 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:37.947722 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:37.949007 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:37.950831 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:37.951913 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:37.952116 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:37.956839 (+ 4723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:37.956847 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:37.956863 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:37.956904 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:37.959900 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:37.960692 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:37.960698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:37.960703 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:37.960871 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:37.961317 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:37.961321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:37.961323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:37.961856 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:37.962112 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:37.962566 (+ 454us) server_negotiation.cc:299] Negotiation successful
0618 22:37:37.962942 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":330,"threads_started":1}
I20260618 22:37:37.964334 5921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:37.966722 5921 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e8b54d61544f70a0486eec4e5661ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41725 } }
I20260618 22:37:37.967186 5921 leader_election.cc:302] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2e8b54d61544f70a0486eec4e5661ff; no voters:
I20260618 22:37:37.967698 5921 leader_election.cc:288] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:37.967825 5926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:37.968463 5926 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [term 1 LEADER]: Becoming Leader. State: Replica: b2e8b54d61544f70a0486eec4e5661ff, State: Running, Role: LEADER
I20260618 22:37:37.968997 5926 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e8b54d61544f70a0486eec4e5661ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41725 } }
I20260618 22:37:37.969533 5921 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:37.972180 5927 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2e8b54d61544f70a0486eec4e5661ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e8b54d61544f70a0486eec4e5661ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41725 } } }
I20260618 22:37:37.972648 5928 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2e8b54d61544f70a0486eec4e5661ff. Latest consensus state: current_term: 1 leader_uuid: "b2e8b54d61544f70a0486eec4e5661ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e8b54d61544f70a0486eec4e5661ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 41725 } } }
I20260618 22:37:37.972965 5927 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:37.973358 5928 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:37.975039 5936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:37.980064 5936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:37.988126 5936 catalog_manager.cc:1347] Generated new cluster ID: 961195de77574f7585f65b284522569c
I20260618 22:37:37.988248 5936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:38.015403 5936 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:38.017280 5936 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:38.024956 5936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff: Generated new TSK 0
I20260618 22:37:38.026041 5936 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.150.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
Entry for principal kudu/127.18.150.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab.
I20260618 22:37:38.082815 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:41725
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:38.393142 5947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:38.393399 5947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:38.393486 5947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:38.400365 5947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:38.400458 5947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:38.400588 5947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:37:38.409793 5947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.129.keytab
--principal=kudu/127.18.150.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:41725
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:38.410955 5947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:38.412206 5947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:38.424256 5953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:38.424327 5952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:38.424297 5955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:38.425000 5947 server_base.cc:1034] running on GCE node
I20260618 22:37:38.425527 5947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:38.426367 5947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:38.427551 5947 hybrid_clock.cc:648] HybridClock initialized: now 1781822258427508 us; error 38 us; skew 500 ppm
I20260618 22:37:38.427820 5947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:38 dist-test-slave-h7jz krb5kdc[5841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822258, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:38.430894 5947 init.cc:377] Logged in from keytab as kudu/127.18.150.129@KRBTEST.COM (short username kudu)
I20260618 22:37:38.432276 5947 webserver.cc:460] Webserver started at http://127.18.150.129:39861/ using document root <none> and password file <none>
I20260618 22:37:38.432911 5947 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:38.433023 5947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:38.433311 5947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:38.436053 5947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "962481926de7438fac70f22d8dd97b8f"
format_stamp: "Formatted at 2026-06-18 22:37:38 on dist-test-slave-h7jz"
server_key: "e1f02b6def7b438f624614068353d816"
server_key_iv: "ab56d8cda0f4f9ecade44595029cd368"
server_key_version: "encryptionkey@0"
I20260618 22:37:38.436756 5947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "962481926de7438fac70f22d8dd97b8f"
format_stamp: "Formatted at 2026-06-18 22:37:38 on dist-test-slave-h7jz"
server_key: "e1f02b6def7b438f624614068353d816"
server_key_iv: "ab56d8cda0f4f9ecade44595029cd368"
server_key_version: "encryptionkey@0"
I20260618 22:37:38.441740 5947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260618 22:37:38.445250 5962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:38.446547 5947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:37:38.446695 5947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "962481926de7438fac70f22d8dd97b8f"
format_stamp: "Formatted at 2026-06-18 22:37:38 on dist-test-slave-h7jz"
server_key: "e1f02b6def7b438f624614068353d816"
server_key_iv: "ab56d8cda0f4f9ecade44595029cd368"
server_key_version: "encryptionkey@0"
I20260618 22:37:38.446884 5947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:38.464977 5947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:38.468766 5947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:38.469018 5947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:38.469884 5947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:38.471244 5947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:38.471318 5947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:38.471433 5947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:38.471483 5947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:38.500819 5947 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:33235
I20260618 22:37:38.500859 6074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:33235 every 8 connection(s)
I20260618 22:37:38.502141 5947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:37:38.503903 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 5947
I20260618 22:37:38.504078 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:37:38.504542 19034 external_mini_cluster.cc:1428] Setting key cbda0147c55169a5486c3e2ca979f23c
Jun 18 22:37:38 dist-test-slave-h7jz krb5kdc[5841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822258, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.129@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:38.521670 6077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:38.504726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:39277 (local address 127.18.150.190:41725)
0618 22:37:38.505178 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:38.505182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:38.506320 (+ 1138us) server_negotiation.cc:407] Connection header received
0618 22:37:38.508205 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:38.508210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:38.508277 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:38.508368 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:38.509881 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:38.510628 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:38.512150 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:38.512554 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:38.512665 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:38.516255 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:38.516260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:38.516263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:38.516296 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:38.518797 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:38.519460 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:38.519465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:38.519469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:38.519598 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:38.519990 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:38.519995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:38.519998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:38.520279 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:38.520409 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:38.521076 (+ 667us) server_negotiation.cc:299] Negotiation successful
0618 22:37:38.521314 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":215,"threads_started":1}
I20260618 22:37:38.522629 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:38.505369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:41725 (local address 127.18.150.129:39277)
0618 22:37:38.506172 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:38.506217 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:38.507893 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:38.508632 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:38.508641 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:38.509148 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:38.509715 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:38.509734 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:38.510776 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:38.510780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:38.512004 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:38.512016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:38.512768 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:38.512776 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:38.512996 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:38.513705 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:37:38.513751 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:38.516017 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:38.518924 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:38.518933 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:38.518950 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:38.519348 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:38.519698 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:38.519704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:38.519708 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:38.519888 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:38.520399 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:38.520406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:38.520740 (+ 334us) client_negotiation.cc:769] Sending connection context
0618 22:37:38.520982 (+ 242us) client_negotiation.cc:240] Negotiation successful
0618 22:37:38.521480 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":316,"threads_started":1}
I20260618 22:37:38.524186 6075 heartbeater.cc:347] Connected to a master server at 127.18.150.190:41725
I20260618 22:37:38.524519 6075 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:38.525444 6075 heartbeater.cc:511] Master 127.18.150.190:41725 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.150.130@KRBTEST.COM; defaulting to no policy
I20260618 22:37:38.527793 5886 ts_manager.cc:194] Registered new tserver with Master: 962481926de7438fac70f22d8dd97b8f (127.18.150.129:33235)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.130@KRBTEST.COM" created.
I20260618 22:37:38.529321 5886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.129@KRBTEST.COM'} at 127.18.150.129:39277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
Entry for principal kudu/127.18.150.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab.
I20260618 22:37:38.554736 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:41725
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:38.865726 6081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:38.866007 6081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:38.866115 6081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:38.873211 6081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:38.873307 6081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:38.873440 6081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:37:38.882884 6081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.130.keytab
--principal=kudu/127.18.150.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:41725
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:38.884047 6081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:38.885315 6081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:38.897579 6086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:38.897720 6089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:38.897787 6087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:38.898389 6081 server_base.cc:1034] running on GCE node
I20260618 22:37:38.898975 6081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:38.899868 6081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:38.901089 6081 hybrid_clock.cc:648] HybridClock initialized: now 1781822258901035 us; error 39 us; skew 500 ppm
I20260618 22:37:38.901378 6081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:38 dist-test-slave-h7jz krb5kdc[5841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822258, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:38.904287 6081 init.cc:377] Logged in from keytab as kudu/127.18.150.130@KRBTEST.COM (short username kudu)
I20260618 22:37:38.905581 6081 webserver.cc:460] Webserver started at http://127.18.150.130:38655/ using document root <none> and password file <none>
I20260618 22:37:38.906234 6081 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:38.906330 6081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:38.906575 6081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:38.909245 6081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "5de75a804c68440caf6cb13c1c37e001"
format_stamp: "Formatted at 2026-06-18 22:37:38 on dist-test-slave-h7jz"
server_key: "33ece8bacefbba060a19a8cbcd9a7287"
server_key_iv: "7d8776996f49758c98eb7e62664f8982"
server_key_version: "encryptionkey@0"
I20260618 22:37:38.909901 6081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "5de75a804c68440caf6cb13c1c37e001"
format_stamp: "Formatted at 2026-06-18 22:37:38 on dist-test-slave-h7jz"
server_key: "33ece8bacefbba060a19a8cbcd9a7287"
server_key_iv: "7d8776996f49758c98eb7e62664f8982"
server_key_version: "encryptionkey@0"
I20260618 22:37:38.914633 6081 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260618 22:37:38.918244 6096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:38.919520 6081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:37:38.919657 6081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "5de75a804c68440caf6cb13c1c37e001"
format_stamp: "Formatted at 2026-06-18 22:37:38 on dist-test-slave-h7jz"
server_key: "33ece8bacefbba060a19a8cbcd9a7287"
server_key_iv: "7d8776996f49758c98eb7e62664f8982"
server_key_version: "encryptionkey@0"
I20260618 22:37:38.919862 6081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:38.939589 6081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:38.943600 6081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:38.943858 6081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:38.944764 6081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:38.946089 6081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:38.946162 6081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:38.946280 6081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:38.946332 6081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:38.974349 6081 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:41107
I20260618 22:37:38.974398 6208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:41107 every 8 connection(s)
I20260618 22:37:38.975667 6081 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:37:38.976907 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 6081
I20260618 22:37:38.977073 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:37:38.977476 19034 external_mini_cluster.cc:1428] Setting key 19c6c290e4d1902c203382e1e7b058ad
Jun 18 22:37:38 dist-test-slave-h7jz krb5kdc[5841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822258, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.130@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:38.995290 6077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:38.978136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:40457 (local address 127.18.150.190:41725)
0618 22:37:38.978310 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:38.978316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:38.979552 (+ 1236us) server_negotiation.cc:407] Connection header received
0618 22:37:38.981497 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:38.981503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:38.981577 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:38.981676 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:38.983170 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:38.984111 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:38.985603 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:38.986180 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:38.986319 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:38.990065 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:38.990070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:38.990073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:38.990104 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:38.992405 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:38.993092 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:38.993095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:38.993098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:38.993241 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:38.993632 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:38.993635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:38.993637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:38.993883 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:38.994038 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:38.994718 (+ 680us) server_negotiation.cc:299] Negotiation successful
0618 22:37:38.994950 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:37:38.996291 6211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:38.978708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:41725 (local address 127.18.150.130:40457)
0618 22:37:38.979427 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:38.979466 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:38.981225 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:38.981907 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:38.981915 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:38.982449 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:38.983016 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:38.983033 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:38.984243 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:38.984247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:38.985469 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:38.985481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:38.986395 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:38.986401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:38.986589 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:38.987285 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:37:38.987322 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:38.989844 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:38.992535 (+ 2691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:38.992545 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:38.992562 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:38.992973 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:38.993343 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:38.993348 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:38.993352 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:38.993525 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:38.994014 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:38.994022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:38.994357 (+ 335us) client_negotiation.cc:769] Sending connection context
0618 22:37:38.994594 (+ 237us) client_negotiation.cc:240] Negotiation successful
0618 22:37:38.995177 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":286,"threads_started":1}
I20260618 22:37:38.997748 6209 heartbeater.cc:347] Connected to a master server at 127.18.150.190:41725
I20260618 22:37:38.998066 6209 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:38.998742 6209 heartbeater.cc:511] Master 127.18.150.190:41725 requested a full tablet report, sending...
I20260618 22:37:39.000136 5886 ts_manager.cc:194] Registered new tserver with Master: 5de75a804c68440caf6cb13c1c37e001 (127.18.150.130:41107)
WARNING: no policy specified for kudu/127.18.150.131@KRBTEST.COM; defaulting to no policy
I20260618 22:37:39.000953 5886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.130@KRBTEST.COM'} at 127.18.150.130:40457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
Entry for principal kudu/127.18.150.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab.
I20260618 22:37:39.029165 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:41725
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:39685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:39.342118 6214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:39.342396 6214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:39.342486 6214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:39.349673 6214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:39.349787 6214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:39.349901 6214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:37:39.359723 6214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:39685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.131.keytab
--principal=kudu/127.18.150.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:41725
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:39.360953 6214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:39.362236 6214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:39.375349 6219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:39.375412 6220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:39.375379 6222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:39.376130 6214 server_base.cc:1034] running on GCE node
I20260618 22:37:39.376683 6214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:39.377609 6214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:39.378806 6214 hybrid_clock.cc:648] HybridClock initialized: now 1781822259378758 us; error 30 us; skew 500 ppm
I20260618 22:37:39.379089 6214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:39 dist-test-slave-h7jz krb5kdc[5841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822259, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:39.382113 6214 init.cc:377] Logged in from keytab as kudu/127.18.150.131@KRBTEST.COM (short username kudu)
I20260618 22:37:39.383569 6214 webserver.cc:460] Webserver started at http://127.18.150.131:35725/ using document root <none> and password file <none>
I20260618 22:37:39.384212 6214 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:39.384315 6214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:39.384579 6214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:39.387583 6214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "f92a8ed083694e7db8571d9baf67ed1d"
format_stamp: "Formatted at 2026-06-18 22:37:39 on dist-test-slave-h7jz"
server_key: "54550ffb9cd89ae1fedee7095a561752"
server_key_iv: "9ac490d3fbb058363a67814e83b907af"
server_key_version: "encryptionkey@0"
I20260618 22:37:39.388259 6214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "f92a8ed083694e7db8571d9baf67ed1d"
format_stamp: "Formatted at 2026-06-18 22:37:39 on dist-test-slave-h7jz"
server_key: "54550ffb9cd89ae1fedee7095a561752"
server_key_iv: "9ac490d3fbb058363a67814e83b907af"
server_key_version: "encryptionkey@0"
I20260618 22:37:39.393144 6214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:37:39.396446 6229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:39.397763 6214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:37:39.397890 6214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "f92a8ed083694e7db8571d9baf67ed1d"
format_stamp: "Formatted at 2026-06-18 22:37:39 on dist-test-slave-h7jz"
server_key: "54550ffb9cd89ae1fedee7095a561752"
server_key_iv: "9ac490d3fbb058363a67814e83b907af"
server_key_version: "encryptionkey@0"
I20260618 22:37:39.398087 6214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:39.414556 6214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:39.419030 6214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:39.419281 6214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:39.420202 6214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:39.421630 6214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:39.421698 6214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:39.421784 6214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:39.421825 6214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:39.451313 6214 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:41863
I20260618 22:37:39.451354 6341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:41863 every 8 connection(s)
I20260618 22:37:39.452538 6214 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:37:39.460911 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 6214
I20260618 22:37:39.461066 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:37:39.461473 19034 external_mini_cluster.cc:1428] Setting key 7e7f25d1b6f2b0cbd4f4cd23707c3d78
Jun 18 22:37:39 dist-test-slave-h7jz krb5kdc[5841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822259, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.131@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:37:39.471102 6077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.454978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:36071 (local address 127.18.150.190:41725)
0618 22:37:39.455139 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:39.455143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:39.456276 (+ 1133us) server_negotiation.cc:407] Connection header received
0618 22:37:39.457999 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:39.458004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:39.458073 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:39.458162 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:39.459574 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.460308 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.461922 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.462449 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.462566 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.465724 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:39.465728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:39.465732 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:39.465763 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:39.468207 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:39.468852 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:39.468857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:39.468861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:39.468981 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:39.469390 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:39.469395 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:39.469399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:39.469713 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:39.469835 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:39.470559 (+ 724us) server_negotiation.cc:299] Negotiation successful
0618 22:37:39.470780 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:37:39.472031 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.455391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:41725 (local address 127.18.150.131:36071)
0618 22:37:39.456066 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:39.456110 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:39.457690 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:39.458354 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:39.458362 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:39.458874 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:39.459423 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.459440 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.460443 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.460446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.461784 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.461796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.462655 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.462663 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.462898 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.463507 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:37:39.463529 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:39.465524 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:39.468331 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:39.468340 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:39.468357 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:39.468737 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:39.469088 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:37:39.469092 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:37:39.469095 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:37:39.469270 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:37:39.469855 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:39.469863 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:39.470204 (+ 341us) client_negotiation.cc:769] Sending connection context
0618 22:37:39.470445 (+ 241us) client_negotiation.cc:240] Negotiation successful
0618 22:37:39.470975 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":251,"threads_started":1}
I20260618 22:37:39.473367 6342 heartbeater.cc:347] Connected to a master server at 127.18.150.190:41725
I20260618 22:37:39.473641 6342 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:39.474462 6342 heartbeater.cc:511] Master 127.18.150.190:41725 requested a full tablet report, sending...
I20260618 22:37:39.475962 5885 ts_manager.cc:194] Registered new tserver with Master: f92a8ed083694e7db8571d9baf67ed1d (127.18.150.131:41863)
I20260618 22:37:39.476766 5885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.150.131@KRBTEST.COM'} at 127.18.150.131:36071
I20260618 22:37:39.487812 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:37:39.503401 6077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.491570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59198 (local address 127.18.150.190:41725)
0618 22:37:39.491728 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:39.491734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:39.492112 (+ 378us) server_negotiation.cc:407] Connection header received
0618 22:37:39.492279 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:39.492283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:39.492344 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:39.492424 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:39.493820 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.494574 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.495828 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.496222 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.496329 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.498533 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:39.498538 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:39.498541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:39.498571 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:39.500864 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:39.501478 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:39.501483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:39.501486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:39.501620 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:39.502014 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:39.502020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:39.502023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:39.502332 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:39.502551 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:39.502904 (+ 353us) server_negotiation.cc:299] Negotiation successful
0618 22:37:39.503105 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:37:39.507174 5885 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:37:39.513689 5885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:37:39.532702 6075 heartbeater.cc:503] Master 127.18.150.190:41725 was elected leader, sending a full tablet report...
I20260618 22:37:39.536386 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.526152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.18.150.129:33235)
0618 22:37:39.527915 (+ 1763us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:39.527921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:39.528503 (+ 582us) server_negotiation.cc:407] Connection header received
0618 22:37:39.528600 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:39.528608 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:39.528839 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:39.529001 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:39.530082 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.531059 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.534136 (+ 3077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.535334 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.535512 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.535686 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:39.535954 (+ 268us) server_negotiation.cc:299] Negotiation successful
0618 22:37:39.536112 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1632,"thread_start_us":1499,"threads_started":1}
I20260618 22:37:39.536423 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.525093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:41863 (local address 127.0.0.1:59242)
0618 22:37:39.526773 (+ 1680us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:39.526833 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:39.527004 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:39.528579 (+ 1575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:39.528586 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:39.528615 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:39.529219 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.529233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.531489 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.531493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.533576 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.533589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.535342 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.535348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.535564 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.535619 (+ 55us) client_negotiation.cc:769] Sending connection context
0618 22:37:39.535737 (+ 118us) client_negotiation.cc:240] Negotiation successful
0618 22:37:39.535944 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1411,"mutex_wait_us":2,"thread_start_us":2925,"threads_started":1}
I20260618 22:37:39.536912 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.525418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59242 (local address 127.18.150.131:41863)
0618 22:37:39.527649 (+ 2231us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:39.527659 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:39.527690 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:37:39.527800 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:39.527808 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:39.528068 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:39.528270 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:39.529572 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.531339 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.533769 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.534851 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.535021 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.535206 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:39.536440 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0618 22:37:39.536601 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2039,"thread_start_us":1878,"threads_started":1}
I20260618 22:37:39.537140 6354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.525574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:33235 (local address 127.0.0.1:49118)
0618 22:37:39.528253 (+ 2679us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:39.528269 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:39.528385 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:39.529252 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:39.529257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:39.529352 (+ 95us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:39.529845 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.529861 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.531176 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.531181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.533837 (+ 2656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.533851 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.535538 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.535548 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.535786 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.535809 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:37:39.535879 (+ 70us) client_negotiation.cc:240] Negotiation successful
0618 22:37:39.536004 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2554,"mutex_wait_us":76,"thread_start_us":799,"threads_started":1}
I20260618 22:37:39.539739 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.529188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:41107 (local address 127.0.0.1:38606)
0618 22:37:39.529873 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:39.529892 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:39.530008 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:39.532312 (+ 2304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:39.532317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:39.532345 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:39.532951 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.532963 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.534871 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.534876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.537463 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.537475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.539009 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.539016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.539175 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.539205 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:37:39.539291 (+ 86us) client_negotiation.cc:240] Negotiation successful
0618 22:37:39.539407 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":25,"thread_start_us":442,"threads_started":1}
I20260618 22:37:39.539817 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.528585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38606 (local address 127.18.150.130:41107)
0618 22:37:39.531534 (+ 2949us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:39.531542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:39.531567 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:37:39.531654 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:39.531662 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:39.531883 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:39.532060 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:39.533243 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.534734 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.537602 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.538807 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.538963 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.539130 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:39.539359 (+ 229us) server_negotiation.cc:299] Negotiation successful
0618 22:37:39.539510 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2790,"thread_start_us":2696,"threads_started":1}
I20260618 22:37:39.540845 6277 tablet_service.cc:1467] Processing CreateTablet for tablet 226fadfd60dc405faf91fb8b58edd797 (DEFAULT_TABLE table=test-table [id=25aec4a8b1cb4440af20518dbb877d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:39.540802 6010 tablet_service.cc:1467] Processing CreateTablet for tablet 226fadfd60dc405faf91fb8b58edd797 (DEFAULT_TABLE table=test-table [id=25aec4a8b1cb4440af20518dbb877d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:39.542299 6010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 226fadfd60dc405faf91fb8b58edd797. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:39.542480 6277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 226fadfd60dc405faf91fb8b58edd797. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:39.543329 6144 tablet_service.cc:1467] Processing CreateTablet for tablet 226fadfd60dc405faf91fb8b58edd797 (DEFAULT_TABLE table=test-table [id=25aec4a8b1cb4440af20518dbb877d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:39.544557 6144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 226fadfd60dc405faf91fb8b58edd797. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:39.553050 6360 tablet_bootstrap.cc:492] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: Bootstrap starting.
I20260618 22:37:39.555892 6361 tablet_bootstrap.cc:492] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f: Bootstrap starting.
I20260618 22:37:39.556945 6360 tablet_bootstrap.cc:654] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:39.558161 6362 tablet_bootstrap.cc:492] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001: Bootstrap starting.
I20260618 22:37:39.558295 6360 log.cc:824] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:39.559231 6361 tablet_bootstrap.cc:654] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:39.560279 6361 log.cc:824] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:39.561152 6362 tablet_bootstrap.cc:654] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:39.561343 6360 tablet_bootstrap.cc:492] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: No bootstrap required, opened a new log
I20260618 22:37:39.561607 6360 ts_tablet_manager.cc:1397] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260618 22:37:39.562152 6362 log.cc:824] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:39.563370 6361 tablet_bootstrap.cc:492] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f: No bootstrap required, opened a new log
I20260618 22:37:39.563617 6361 ts_tablet_manager.cc:1397] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260618 22:37:39.564821 6362 tablet_bootstrap.cc:492] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001: No bootstrap required, opened a new log
I20260618 22:37:39.565060 6362 ts_tablet_manager.cc:1397] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260618 22:37:39.571967 6362 raft_consensus.cc:348] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92a8ed083694e7db8571d9baf67ed1d" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 41863 } } peers { permanent_uuid: "5de75a804c68440caf6cb13c1c37e001" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41107 } } peers { permanent_uuid: "962481926de7438fac70f22d8dd97b8f" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 33235 } }
I20260618 22:37:39.571883 6360 raft_consensus.cc:348] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92a8ed083694e7db8571d9baf67ed1d" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 41863 } } peers { permanent_uuid: "5de75a804c68440caf6cb13c1c37e001" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41107 } } peers { permanent_uuid: "962481926de7438fac70f22d8dd97b8f" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 33235 } }
I20260618 22:37:39.572302 6362 raft_consensus.cc:374] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:39.572307 6360 raft_consensus.cc:374] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:39.572393 6360 raft_consensus.cc:729] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f92a8ed083694e7db8571d9baf67ed1d, State: Initialized, Role: FOLLOWER
I20260618 22:37:39.572405 6362 raft_consensus.cc:729] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de75a804c68440caf6cb13c1c37e001, State: Initialized, Role: FOLLOWER
I20260618 22:37:39.572942 6362 consensus_queue.cc:260] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92a8ed083694e7db8571d9baf67ed1d" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 41863 } } peers { permanent_uuid: "5de75a804c68440caf6cb13c1c37e001" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41107 } } peers { permanent_uuid: "962481926de7438fac70f22d8dd97b8f" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 33235 } }
I20260618 22:37:39.573035 6360 consensus_queue.cc:260] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92a8ed083694e7db8571d9baf67ed1d" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 41863 } } peers { permanent_uuid: "5de75a804c68440caf6cb13c1c37e001" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41107 } } peers { permanent_uuid: "962481926de7438fac70f22d8dd97b8f" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 33235 } }
I20260618 22:37:39.573655 6361 raft_consensus.cc:348] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92a8ed083694e7db8571d9baf67ed1d" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 41863 } } peers { permanent_uuid: "5de75a804c68440caf6cb13c1c37e001" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41107 } } peers { permanent_uuid: "962481926de7438fac70f22d8dd97b8f" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 33235 } }
I20260618 22:37:39.574149 6361 raft_consensus.cc:374] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:39.574333 6361 raft_consensus.cc:729] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 962481926de7438fac70f22d8dd97b8f, State: Initialized, Role: FOLLOWER
I20260618 22:37:39.575044 6361 consensus_queue.cc:260] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92a8ed083694e7db8571d9baf67ed1d" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 41863 } } peers { permanent_uuid: "5de75a804c68440caf6cb13c1c37e001" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41107 } } peers { permanent_uuid: "962481926de7438fac70f22d8dd97b8f" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 33235 } }
I20260618 22:37:39.575362 6209 heartbeater.cc:503] Master 127.18.150.190:41725 was elected leader, sending a full tablet report...
I20260618 22:37:39.576054 6362 ts_tablet_manager.cc:1428] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
I20260618 22:37:39.577451 6342 heartbeater.cc:503] Master 127.18.150.190:41725 was elected leader, sending a full tablet report...
I20260618 22:37:39.578730 6361 ts_tablet_manager.cc:1428] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260618 22:37:39.579417 6360 ts_tablet_manager.cc:1428] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
I20260618 22:37:39.628190 6367 raft_consensus.cc:482] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:37:39.628479 6367 raft_consensus.cc:504] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92a8ed083694e7db8571d9baf67ed1d" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 41863 } } peers { permanent_uuid: "5de75a804c68440caf6cb13c1c37e001" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41107 } } peers { permanent_uuid: "962481926de7438fac70f22d8dd97b8f" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 33235 } }
I20260618 22:37:39.630165 6367 leader_election.cc:288] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5de75a804c68440caf6cb13c1c37e001 (127.18.150.130:41107), 962481926de7438fac70f22d8dd97b8f (127.18.150.129:33235)
I20260618 22:37:39.636467 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.630443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:33235 (local address 127.18.150.131:35115)
0618 22:37:39.630776 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:39.630800 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:39.630936 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:39.631332 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:39.631335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:39.631363 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:39.631515 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.631522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.632938 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.632944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.634501 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.634513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.635724 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.635731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.635909 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.635951 (+ 42us) client_negotiation.cc:769] Sending connection context
0618 22:37:39.636042 (+ 91us) client_negotiation.cc:240] Negotiation successful
0618 22:37:39.636187 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":58}
I20260618 22:37:39.636752 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.630667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:35767 (local address 127.18.150.130:41107)
0618 22:37:39.630888 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:39.630893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:39.631484 (+ 591us) server_negotiation.cc:407] Connection header received
0618 22:37:39.631640 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:39.631644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:39.631701 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:39.631772 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:39.632316 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.633176 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.634963 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.635788 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.635930 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.635996 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:39.636330 (+ 334us) server_negotiation.cc:299] Negotiation successful
0618 22:37:39.636487 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260618 22:37:39.636801 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.630662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:35115 (local address 127.18.150.129:33235)
0618 22:37:39.630868 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:39.630874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:39.630896 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:37:39.631095 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:39.631100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:39.631181 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:39.631290 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:39.631942 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.632804 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.634650 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.635561 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.635683 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.635738 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:39.636142 (+ 404us) server_negotiation.cc:299] Negotiation successful
0618 22:37:39.636326 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:37:39.637377 6369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.630518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:41107 (local address 127.18.150.131:35767)
0618 22:37:39.631372 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:39.631393 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:39.631518 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:39.631796 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:39.631800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:39.631821 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:39.632157 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.632168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.633312 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.633316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.634816 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:39.634830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.635961 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.635968 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:39.636124 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.636157 (+ 33us) client_negotiation.cc:769] Sending connection context
0618 22:37:39.636936 (+ 779us) client_negotiation.cc:240] Negotiation successful
0618 22:37:39.637055 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":17,"thread_start_us":507,"threads_started":1}
I20260618 22:37:39.637730 6030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "226fadfd60dc405faf91fb8b58edd797" candidate_uuid: "f92a8ed083694e7db8571d9baf67ed1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "962481926de7438fac70f22d8dd97b8f" is_pre_election: true
I20260618 22:37:39.638336 6030 raft_consensus.cc:2454] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f92a8ed083694e7db8571d9baf67ed1d in term 0.
I20260618 22:37:39.638348 6164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "226fadfd60dc405faf91fb8b58edd797" candidate_uuid: "f92a8ed083694e7db8571d9baf67ed1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de75a804c68440caf6cb13c1c37e001" is_pre_election: true
I20260618 22:37:39.638866 6164 raft_consensus.cc:2454] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f92a8ed083694e7db8571d9baf67ed1d in term 0.
I20260618 22:37:39.639055 6233 leader_election.cc:302] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 962481926de7438fac70f22d8dd97b8f, f92a8ed083694e7db8571d9baf67ed1d; no voters:
I20260618 22:37:39.639444 6367 raft_consensus.cc:2781] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:37:39.639577 6367 raft_consensus.cc:482] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:37:39.639674 6367 raft_consensus.cc:3037] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:39.641809 6367 raft_consensus.cc:504] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92a8ed083694e7db8571d9baf67ed1d" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 41863 } } peers { permanent_uuid: "5de75a804c68440caf6cb13c1c37e001" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41107 } } peers { permanent_uuid: "962481926de7438fac70f22d8dd97b8f" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 33235 } }
I20260618 22:37:39.642405 6367 leader_election.cc:288] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [CANDIDATE]: Term 1 election: Requested vote from peers 5de75a804c68440caf6cb13c1c37e001 (127.18.150.130:41107), 962481926de7438fac70f22d8dd97b8f (127.18.150.129:33235)
I20260618 22:37:39.642870 6164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "226fadfd60dc405faf91fb8b58edd797" candidate_uuid: "f92a8ed083694e7db8571d9baf67ed1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de75a804c68440caf6cb13c1c37e001"
I20260618 22:37:39.642933 6030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "226fadfd60dc405faf91fb8b58edd797" candidate_uuid: "f92a8ed083694e7db8571d9baf67ed1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "962481926de7438fac70f22d8dd97b8f"
I20260618 22:37:39.643072 6164 raft_consensus.cc:3037] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:39.643110 6030 raft_consensus.cc:3037] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:39.646225 6164 raft_consensus.cc:2454] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f92a8ed083694e7db8571d9baf67ed1d in term 1.
I20260618 22:37:39.646212 6030 raft_consensus.cc:2454] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f92a8ed083694e7db8571d9baf67ed1d in term 1.
I20260618 22:37:39.646725 6230 leader_election.cc:302] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de75a804c68440caf6cb13c1c37e001, f92a8ed083694e7db8571d9baf67ed1d; no voters:
I20260618 22:37:39.647068 6367 raft_consensus.cc:2781] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:39.647522 6367 raft_consensus.cc:686] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 1 LEADER]: Becoming Leader. State: Replica: f92a8ed083694e7db8571d9baf67ed1d, State: Running, Role: LEADER
I20260618 22:37:39.648041 6367 consensus_queue.cc:237] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92a8ed083694e7db8571d9baf67ed1d" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 41863 } } peers { permanent_uuid: "5de75a804c68440caf6cb13c1c37e001" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41107 } } peers { permanent_uuid: "962481926de7438fac70f22d8dd97b8f" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 33235 } }
I20260618 22:37:39.652683 5886 catalog_manager.cc:5495] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d reported cstate change: term changed from 0 to 1, leader changed from <none> to f92a8ed083694e7db8571d9baf67ed1d (127.18.150.131). New cstate: current_term: 1 leader_uuid: "f92a8ed083694e7db8571d9baf67ed1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f92a8ed083694e7db8571d9baf67ed1d" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 41863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5de75a804c68440caf6cb13c1c37e001" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "962481926de7438fac70f22d8dd97b8f" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 33235 } health_report { overall_health: UNKNOWN } } }
I20260618 22:37:39.688766 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.683592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59244 (local address 127.18.150.131:41863)
0618 22:37:39.683764 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:39.683768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:39.683901 (+ 133us) server_negotiation.cc:407] Connection header received
0618 22:37:39.684069 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:39.684073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:39.684151 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:39.684271 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:37:39.684576 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.685277 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.686702 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.687101 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.687238 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.687694 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:37:39.688190 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:37:39.688342 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:39.688432 (+ 90us) server_negotiation.cc:299] Negotiation successful
0618 22:37:39.688515 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:37:39.697170 6030 raft_consensus.cc:1261] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f [term 1 FOLLOWER]: Refusing update from remote peer f92a8ed083694e7db8571d9baf67ed1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:39.697170 6164 raft_consensus.cc:1261] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001 [term 1 FOLLOWER]: Refusing update from remote peer f92a8ed083694e7db8571d9baf67ed1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:39.698287 6370 consensus_queue.cc:1035] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "962481926de7438fac70f22d8dd97b8f" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 33235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:37:39.698652 6367 consensus_queue.cc:1035] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5de75a804c68440caf6cb13c1c37e001" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:37:39.703969 6343 tablet.cc:2369] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:37:39.708652 6374 mvcc.cc:205] Tried to move back new op lower bound from 7298343975708831744 to 7298343975522402304. Current Snapshot: MvccSnapshot[applied={T|T < 7298343975708831744}]
I20260618 22:37:39.721036 5886 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:39.721287 5886 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:39.724622 5886 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2e8b54d61544f70a0486eec4e5661ff: Sending DeleteTablet for 3 replicas of tablet 226fadfd60dc405faf91fb8b58edd797
I20260618 22:37:39.725927 6277 tablet_service.cc:1514] Processing DeleteTablet for tablet 226fadfd60dc405faf91fb8b58edd797 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:39 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:59242
I20260618 22:37:39.726547 6144 tablet_service.cc:1514] Processing DeleteTablet for tablet 226fadfd60dc405faf91fb8b58edd797 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:39 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:38606
I20260618 22:37:39.729290 6010 tablet_service.cc:1514] Processing DeleteTablet for tablet 226fadfd60dc405faf91fb8b58edd797 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:39 UTC) from {username='kudu', principal='kudu/127.18.150.190@KRBTEST.COM'} at 127.0.0.1:49118
I20260618 22:37:39.729650 6383 tablet_replica.cc:330] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001: stopping tablet replica
I20260618 22:37:39.730098 6383 raft_consensus.cc:2229] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:37:39.730604 6383 raft_consensus.cc:2258] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:39.731479 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 5947
I20260618 22:37:39.732792 6383 ts_tablet_manager.cc:1905] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:39.739502 6383 ts_tablet_manager.cc:1918] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:39.739670 6383 log.cc:1196] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/226fadfd60dc405faf91fb8b58edd797
I20260618 22:37:39.740180 6383 ts_tablet_manager.cc:1939] T 226fadfd60dc405faf91fb8b58edd797 P 5de75a804c68440caf6cb13c1c37e001: Deleting consensus metadata
I20260618 22:37:39.742223 5870 catalog_manager.cc:4841] TS 5de75a804c68440caf6cb13c1c37e001 (127.18.150.130:41107): tablet 226fadfd60dc405faf91fb8b58edd797 (table test-table [id=25aec4a8b1cb4440af20518dbb877d64]) successfully deleted
I20260618 22:37:39.742411 6382 tablet_replica.cc:330] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: stopping tablet replica
I20260618 22:37:39.742870 6382 raft_consensus.cc:2229] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:37:39.743551 6382 raft_consensus.cc:2258] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:39.746951 6382 ts_tablet_manager.cc:1905] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:39.754637 6382 ts_tablet_manager.cc:1918] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:39.754896 6382 log.cc:1196] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/226fadfd60dc405faf91fb8b58edd797
I20260618 22:37:39.755473 6382 ts_tablet_manager.cc:1939] T 226fadfd60dc405faf91fb8b58edd797 P f92a8ed083694e7db8571d9baf67ed1d: Deleting consensus metadata
I20260618 22:37:39.756196 6384 tablet_replica.cc:330] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f: stopping tablet replica
I20260618 22:37:39.756652 6384 raft_consensus.cc:2229] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:37:39.757208 6384 raft_consensus.cc:2258] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:39.757390 5870 catalog_manager.cc:4841] TS f92a8ed083694e7db8571d9baf67ed1d (127.18.150.131:41863): tablet 226fadfd60dc405faf91fb8b58edd797 (table test-table [id=25aec4a8b1cb4440af20518dbb877d64]) successfully deleted
I20260618 22:37:39.759119 6384 ts_tablet_manager.cc:1905] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 18 22:37:39 dist-test-slave-h7jz krb5kdc[5841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.129@KRBTEST.COM
I20260618 22:37:39.764415 6384 ts_tablet_manager.cc:1918] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:39.764585 6384 log.cc:1196] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821960891835-19034-0/minicluster-data/ts-0/wal/wals/226fadfd60dc405faf91fb8b58edd797
I20260618 22:37:39.765110 6384 ts_tablet_manager.cc:1939] T 226fadfd60dc405faf91fb8b58edd797 P 962481926de7438fac70f22d8dd97b8f: Deleting consensus metadata
I20260618 22:37:39.766737 5873 catalog_manager.cc:4841] TS 962481926de7438fac70f22d8dd97b8f (127.18.150.129:33235): tablet 226fadfd60dc405faf91fb8b58edd797 (table test-table [id=25aec4a8b1cb4440af20518dbb877d64]) successfully deleted
I20260618 22:37:39.769698 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.732825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49126 (local address 127.18.150.129:33235)
0618 22:37:39.733029 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:39.733036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:39.750051 (+ 17015us) server_negotiation.cc:407] Connection header received
0618 22:37:39.750594 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:39.750599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:39.750690 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:39.750767 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:39.752616 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.754024 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.756448 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.757161 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.757304 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.762855 (+ 5551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:39.762884 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:39.762902 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:39.762957 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:39.766873 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:39.767725 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:39.767732 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:39.767735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:39.767876 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:39.768280 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:39.768284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:39.768287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:39.768635 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:39.768874 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:39.769208 (+ 334us) server_negotiation.cc:299] Negotiation successful
0618 22:37:39.769422 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:37:39.770422 6070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:39.857460 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 5947
I20260618 22:37:39.877400 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 6081
Jun 18 22:37:39 dist-test-slave-h7jz krb5kdc[5841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.130@KRBTEST.COM
I20260618 22:37:39.893730 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.878158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.18.150.130:41107)
0618 22:37:39.878347 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:39.878351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:39.878370 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:37:39.878574 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:39.878578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:39.878640 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:39.878719 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:39.880290 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.880974 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.882838 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:39.883216 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:39.883331 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:39.888347 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:39.888361 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:39.888376 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:39.888413 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:39.891191 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:39.891912 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:39.891918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:39.891920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:39.892027 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:39.892401 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:39.892403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:39.892405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:39.892744 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:39.892956 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:39.893240 (+ 284us) server_negotiation.cc:299] Negotiation successful
0618 22:37:39.893403 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260618 22:37:39.894397 6204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:39.982371 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 6081
I20260618 22:37:39.996426 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 6214
Jun 18 22:37:40 dist-test-slave-h7jz krb5kdc[5841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.131@KRBTEST.COM
I20260618 22:37:40.012935 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:39.997160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59256 (local address 127.18.150.131:41863)
0618 22:37:39.997343 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:39.997349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:39.997493 (+ 144us) server_negotiation.cc:407] Connection header received
0618 22:37:39.997674 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:39.997678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:39.997748 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:39.997846 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:39.999183 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:40.000005 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:40.001359 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:40.001860 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:40.002116 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:40.007097 (+ 4981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:40.007110 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:40.007125 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:40.007169 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:40.010187 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:40.010871 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:40.010878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:40.010881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:40.011046 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:40.011406 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:40.011410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:40.011412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:40.011743 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:40.012039 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:40.012346 (+ 307us) server_negotiation.cc:299] Negotiation successful
0618 22:37:40.012561 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:37:40.013554 6337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:40.132493 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 6214
I20260618 22:37:40.147497 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 5854
I20260618 22:37:40.148576 5916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:40.255959 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 5854
2026-06-18T22:37:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (3991 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-18T22:37:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:40Z Disabled control of system clock
I20260618 22:37:40.323347 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:37159
--webserver_interface=127.18.150.190
--webserver_port=0
--builtin_ntp_servers=127.18.150.148:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:37159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:37:40.635581 6393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:40.635857 6393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:40.635942 6393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:40.643101 6393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:40.643204 6393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:40.643254 6393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:40.643308 6393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:40.643350 6393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:40.652411 6393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:37159
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:37159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:40.653617 6393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:40.655730 6393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:40.667116 6399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:40.667095 6401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:40.667201 6398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:40.667654 6393 server_base.cc:1034] running on GCE node
I20260618 22:37:40.668352 6393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:40.669688 6393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:40.670884 6393 hybrid_clock.cc:648] HybridClock initialized: now 1781822260670826 us; error 56 us; skew 500 ppm
I20260618 22:37:40.671209 6393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:40.673403 6393 webserver.cc:460] Webserver started at http://127.18.150.190:41693/ using document root <none> and password file <none>
I20260618 22:37:40.674242 6393 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:40.674329 6393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:40.674724 6393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:40.677486 6393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "c4852883bf524b3fbad9dcdc5cbbd445"
format_stamp: "Formatted at 2026-06-18 22:37:40 on dist-test-slave-h7jz"
server_key: "72867b9de7f4690a062ee7ff14e28c71"
server_key_iv: "2aaf96ac42f37366cf1bca69e76c1c0e"
server_key_version: "encryptionkey@0"
I20260618 22:37:40.678272 6393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "c4852883bf524b3fbad9dcdc5cbbd445"
format_stamp: "Formatted at 2026-06-18 22:37:40 on dist-test-slave-h7jz"
server_key: "72867b9de7f4690a062ee7ff14e28c71"
server_key_iv: "2aaf96ac42f37366cf1bca69e76c1c0e"
server_key_version: "encryptionkey@0"
I20260618 22:37:40.683277 6393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260618 22:37:40.686617 6407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:40.687986 6393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:37:40.688146 6393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "c4852883bf524b3fbad9dcdc5cbbd445"
format_stamp: "Formatted at 2026-06-18 22:37:40 on dist-test-slave-h7jz"
server_key: "72867b9de7f4690a062ee7ff14e28c71"
server_key_iv: "2aaf96ac42f37366cf1bca69e76c1c0e"
server_key_version: "encryptionkey@0"
I20260618 22:37:40.688341 6393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:40.699743 6393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:40.700505 6393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:40.700762 6393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:40.717234 6393 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:37159
I20260618 22:37:40.717283 6458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:37159 every 8 connection(s)
I20260618 22:37:40.718564 6393 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:40.722301 6459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:40.723989 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 6393
I20260618 22:37:40.724159 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:40.724684 19034 external_mini_cluster.cc:1428] Setting key 58ac51b7cdde43202c04cdd53ec8a65b
I20260618 22:37:40.734079 6459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445: Bootstrap starting.
I20260618 22:37:40.737479 6459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:40.738170 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:40.726641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57384 (local address 127.18.150.190:37159)
0618 22:37:40.727662 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:40.727680 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:40.727729 (+ 49us) server_negotiation.cc:407] Connection header received
0618 22:37:40.729371 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:40.729392 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:40.729849 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:40.730193 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:40.730750 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:40.732037 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:40.733837 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:40.734624 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:40.734770 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:40.735882 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:40.735891 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:40.735904 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:40.736420 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:40.736505 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:40.736525 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:40.736611 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:40.736854 (+ 243us) server_negotiation.cc:299] Negotiation successful
0618 22:37:40.737139 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":372,"threads_started":1}
I20260618 22:37:40.738601 6459 log.cc:824] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:40.741302 6459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445: No bootstrap required, opened a new log
I20260618 22:37:40.748477 6459 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4852883bf524b3fbad9dcdc5cbbd445" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 37159 } }
I20260618 22:37:40.748852 6459 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:40.748988 6459 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4852883bf524b3fbad9dcdc5cbbd445, State: Initialized, Role: FOLLOWER
I20260618 22:37:40.749504 6459 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4852883bf524b3fbad9dcdc5cbbd445" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 37159 } }
I20260618 22:37:40.749708 6459 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:40.749825 6459 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:40.750008 6459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:40.751997 6459 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4852883bf524b3fbad9dcdc5cbbd445" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 37159 } }
I20260618 22:37:40.752455 6459 leader_election.cc:302] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4852883bf524b3fbad9dcdc5cbbd445; no voters:
I20260618 22:37:40.752985 6459 leader_election.cc:288] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:40.753125 6464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:40.753769 6464 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [term 1 LEADER]: Becoming Leader. State: Replica: c4852883bf524b3fbad9dcdc5cbbd445, State: Running, Role: LEADER
I20260618 22:37:40.754279 6464 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4852883bf524b3fbad9dcdc5cbbd445" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 37159 } }
I20260618 22:37:40.754830 6459 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:40.756887 6466 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4852883bf524b3fbad9dcdc5cbbd445. Latest consensus state: current_term: 1 leader_uuid: "c4852883bf524b3fbad9dcdc5cbbd445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4852883bf524b3fbad9dcdc5cbbd445" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 37159 } } }
I20260618 22:37:40.757340 6466 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:40.757421 6465 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4852883bf524b3fbad9dcdc5cbbd445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4852883bf524b3fbad9dcdc5cbbd445" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 37159 } } }
I20260618 22:37:40.757710 6465 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:40.759106 6473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:40.763489 6473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:40.771157 6473 catalog_manager.cc:1347] Generated new cluster ID: 5e266f7e00fe41fca62345bb00600e79
I20260618 22:37:40.771263 6473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:40.784277 6473 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:40.785480 6473 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:40.797423 6473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445: Generated new TSK 0
I20260618 22:37:40.798295 6473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:40.811182 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.129:0
--local_ip_for_outbound_sockets=127.18.150.129
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:37159
--builtin_ntp_servers=127.18.150.148:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:37:41.129277 6483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:41.129537 6483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:41.129623 6483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:41.136838 6483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:41.136940 6483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:41.137043 6483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.129
I20260618 22:37:41.146106 6483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.150.129
--webserver_port=0
--tserver_master_addrs=127.18.150.190:37159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.129
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:41.147293 6483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:41.148962 6483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:41.161418 6491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:41.161459 6489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:41.161479 6488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:41.162194 6483 server_base.cc:1034] running on GCE node
I20260618 22:37:41.162798 6483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:41.163686 6483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:41.164906 6483 hybrid_clock.cc:648] HybridClock initialized: now 1781822261164854 us; error 38 us; skew 500 ppm
I20260618 22:37:41.165215 6483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:41.167331 6483 webserver.cc:460] Webserver started at http://127.18.150.129:32851/ using document root <none> and password file <none>
I20260618 22:37:41.167976 6483 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:41.168074 6483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:41.168366 6483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:41.171156 6483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/data/instance:
uuid: "51364d1b942647edb9cae3b9c09d4bcc"
format_stamp: "Formatted at 2026-06-18 22:37:41 on dist-test-slave-h7jz"
server_key: "fa1f82eb03147ba380ca851b42c46db8"
server_key_iv: "704d791201f5840bd1232e9d53b299de"
server_key_version: "encryptionkey@0"
I20260618 22:37:41.171873 6483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance:
uuid: "51364d1b942647edb9cae3b9c09d4bcc"
format_stamp: "Formatted at 2026-06-18 22:37:41 on dist-test-slave-h7jz"
server_key: "fa1f82eb03147ba380ca851b42c46db8"
server_key_iv: "704d791201f5840bd1232e9d53b299de"
server_key_version: "encryptionkey@0"
I20260618 22:37:41.176704 6483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:37:41.179883 6497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:41.181120 6483 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:37:41.181288 6483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/wal
uuid: "51364d1b942647edb9cae3b9c09d4bcc"
format_stamp: "Formatted at 2026-06-18 22:37:41 on dist-test-slave-h7jz"
server_key: "fa1f82eb03147ba380ca851b42c46db8"
server_key_iv: "704d791201f5840bd1232e9d53b299de"
server_key_version: "encryptionkey@0"
I20260618 22:37:41.181475 6483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:41.198035 6483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:41.198817 6483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:41.199093 6483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:41.200022 6483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:41.201416 6483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:41.201514 6483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:41.201619 6483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:41.201690 6483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:41.231424 6483 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.129:41527
I20260618 22:37:41.231465 6609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.129:41527 every 8 connection(s)
I20260618 22:37:41.232704 6483 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/data/info.pb
I20260618 22:37:41.233911 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 6483
I20260618 22:37:41.234114 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-0/wal/instance
I20260618 22:37:41.234654 19034 external_mini_cluster.cc:1428] Setting key d035a8c1293e5189aae0af3168ee4792
I20260618 22:37:41.237708 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.130:0
--local_ip_for_outbound_sockets=127.18.150.130
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:37159
--builtin_ntp_servers=127.18.150.148:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:37:41.245067 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:41.235551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.129:44349 (local address 127.18.150.190:37159)
0618 22:37:41.235726 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:41.235731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:41.237512 (+ 1781us) server_negotiation.cc:407] Connection header received
0618 22:37:41.239022 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:41.239027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:41.239102 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:41.239184 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:41.240269 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:41.240949 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:41.242504 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:41.242898 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:41.243014 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:41.244340 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:41.244347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:41.244352 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:41.244374 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:41.244413 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:41.244428 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:41.244517 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:41.244726 (+ 209us) server_negotiation.cc:299] Negotiation successful
0618 22:37:41.244823 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:37:41.246102 6612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:41.236004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:37159 (local address 127.18.150.129:44349)
0618 22:37:41.236978 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:41.237014 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:41.238755 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:41.239429 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:41.239437 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:41.239525 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:41.240108 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:41.240126 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:41.241099 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:41.241102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:41.242371 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:41.242383 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:41.243108 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:41.243115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:41.243338 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:41.243992 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:37:41.243995 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:41.244032 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:37:41.244107 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:41.244535 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:41.244544 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:41.244550 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:37:41.244694 (+ 144us) client_negotiation.cc:240] Negotiation successful
0618 22:37:41.245004 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":313,"threads_started":1}
I20260618 22:37:41.247601 6610 heartbeater.cc:347] Connected to a master server at 127.18.150.190:37159
I20260618 22:37:41.247961 6610 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:41.248919 6610 heartbeater.cc:511] Master 127.18.150.190:37159 requested a full tablet report, sending...
I20260618 22:37:41.251050 6424 ts_manager.cc:194] Registered new tserver with Master: 51364d1b942647edb9cae3b9c09d4bcc (127.18.150.129:41527)
I20260618 22:37:41.252527 6424 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.129:44349
W20260618 22:37:41.550064 6613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:41.550340 6613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:41.550463 6613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:41.557822 6613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:41.557924 6613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:41.558094 6613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.130
I20260618 22:37:41.567044 6613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.150.130
--webserver_port=0
--tserver_master_addrs=127.18.150.190:37159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.130
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:41.568341 6613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:41.570122 6613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:41.582604 6619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:41.582546 6621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:41.582722 6618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:41.583171 6613 server_base.cc:1034] running on GCE node
I20260618 22:37:41.583731 6613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:41.584649 6613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:41.585848 6613 hybrid_clock.cc:648] HybridClock initialized: now 1781822261585784 us; error 59 us; skew 500 ppm
I20260618 22:37:41.586161 6613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:41.588300 6613 webserver.cc:460] Webserver started at http://127.18.150.130:46689/ using document root <none> and password file <none>
I20260618 22:37:41.588955 6613 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:41.589037 6613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:41.589325 6613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:41.592242 6613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/data/instance:
uuid: "f008c691fc2c4b148b54b0c9f7481298"
format_stamp: "Formatted at 2026-06-18 22:37:41 on dist-test-slave-h7jz"
server_key: "465d491d8a8f02bffe29a54b79b78f20"
server_key_iv: "cc50d8cbc06602a09457e48334f3cb1a"
server_key_version: "encryptionkey@0"
I20260618 22:37:41.593053 6613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance:
uuid: "f008c691fc2c4b148b54b0c9f7481298"
format_stamp: "Formatted at 2026-06-18 22:37:41 on dist-test-slave-h7jz"
server_key: "465d491d8a8f02bffe29a54b79b78f20"
server_key_iv: "cc50d8cbc06602a09457e48334f3cb1a"
server_key_version: "encryptionkey@0"
I20260618 22:37:41.597992 6613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260618 22:37:41.601366 6627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:41.602756 6613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:37:41.602895 6613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/wal
uuid: "f008c691fc2c4b148b54b0c9f7481298"
format_stamp: "Formatted at 2026-06-18 22:37:41 on dist-test-slave-h7jz"
server_key: "465d491d8a8f02bffe29a54b79b78f20"
server_key_iv: "cc50d8cbc06602a09457e48334f3cb1a"
server_key_version: "encryptionkey@0"
I20260618 22:37:41.603091 6613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:41.617552 6613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:41.618458 6613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:41.618737 6613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:41.619709 6613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:41.621192 6613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:41.621287 6613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:41.621397 6613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:41.621462 6613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:41.651852 6613 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.130:41809
I20260618 22:37:41.651894 6739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.130:41809 every 8 connection(s)
I20260618 22:37:41.653121 6613 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/data/info.pb
I20260618 22:37:41.662160 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 6613
I20260618 22:37:41.662349 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/wal/instance
I20260618 22:37:41.662917 19034 external_mini_cluster.cc:1428] Setting key 6c776337a0a52895d4038f61539da50a
I20260618 22:37:41.665897 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.150.131:0
--local_ip_for_outbound_sockets=127.18.150.131
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:37159
--builtin_ntp_servers=127.18.150.148:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:37:41.666786 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:41.655572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.130:32911 (local address 127.18.150.190:37159)
0618 22:37:41.655792 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:41.655799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:41.657180 (+ 1381us) server_negotiation.cc:407] Connection header received
0618 22:37:41.659220 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:41.659227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:41.659340 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:41.659448 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:41.660985 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:41.661702 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:41.663862 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:41.664313 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:41.664428 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:41.665905 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:41.665910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:41.665913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:41.665931 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:41.665986 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:41.666005 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:41.666093 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:41.666350 (+ 257us) server_negotiation.cc:299] Negotiation successful
0618 22:37:41.666456 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":77}
I20260618 22:37:41.667793 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:41.656157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:37159 (local address 127.18.150.130:32911)
0618 22:37:41.657038 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:41.657078 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:41.658935 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:41.659874 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:41.659886 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:41.660009 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:41.660788 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:41.660813 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:41.661846 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:41.661851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:41.663712 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:41.663725 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:41.664536 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:41.664544 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:41.664756 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:41.665545 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:37:41.665549 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:41.665604 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:37:41.665709 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:41.666146 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:41.666158 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:41.666164 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:37:41.666321 (+ 157us) client_negotiation.cc:240] Negotiation successful
0618 22:37:41.666733 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":251,"threads_started":1}
I20260618 22:37:41.671324 6740 heartbeater.cc:347] Connected to a master server at 127.18.150.190:37159
I20260618 22:37:41.671615 6740 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:41.672369 6740 heartbeater.cc:511] Master 127.18.150.190:37159 requested a full tablet report, sending...
I20260618 22:37:41.673885 6424 ts_manager.cc:194] Registered new tserver with Master: f008c691fc2c4b148b54b0c9f7481298 (127.18.150.130:41809)
I20260618 22:37:41.674996 6424 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.130:32911
W20260618 22:37:41.983103 6743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:41.983377 6743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:41.983506 6743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:41.990600 6743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:41.990696 6743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:41.990836 6743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.150.131
I20260618 22:37:41.999642 6743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.150.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.150.131
--webserver_port=0
--tserver_master_addrs=127.18.150.190:37159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.150.131
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:42.000829 6743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:42.002542 6743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:42.015102 6748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:42.015149 6751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:42.015070 6749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:42.015799 6743 server_base.cc:1034] running on GCE node
I20260618 22:37:42.016386 6743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:42.017294 6743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:42.018501 6743 hybrid_clock.cc:648] HybridClock initialized: now 1781822262018460 us; error 36 us; skew 500 ppm
I20260618 22:37:42.018827 6743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260618 22:37:42.021029 6743 webserver.cc:460] Webserver started at http://127.18.150.131:41179/ using document root <none> and password file <none>
I20260618 22:37:42.021689 6743 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:42.021786 6743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:42.022132 6743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:42.024924 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/data/instance:
uuid: "518aee346e56405bb294aafdfb0688c2"
format_stamp: "Formatted at 2026-06-18 22:37:42 on dist-test-slave-h7jz"
server_key: "7076c105d98147ea223fae114b8fef80"
server_key_iv: "5f46b932037c53fbc8cc5801b30f0ce0"
server_key_version: "encryptionkey@0"
I20260618 22:37:42.025696 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance:
uuid: "518aee346e56405bb294aafdfb0688c2"
format_stamp: "Formatted at 2026-06-18 22:37:42 on dist-test-slave-h7jz"
server_key: "7076c105d98147ea223fae114b8fef80"
server_key_iv: "5f46b932037c53fbc8cc5801b30f0ce0"
server_key_version: "encryptionkey@0"
I20260618 22:37:42.030987 6743 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260618 22:37:42.034366 6757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:42.035687 6743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:37:42.035838 6743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/wal
uuid: "518aee346e56405bb294aafdfb0688c2"
format_stamp: "Formatted at 2026-06-18 22:37:42 on dist-test-slave-h7jz"
server_key: "7076c105d98147ea223fae114b8fef80"
server_key_iv: "5f46b932037c53fbc8cc5801b30f0ce0"
server_key_version: "encryptionkey@0"
I20260618 22:37:42.036028 6743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:42.072566 6743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:42.073402 6743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:42.073678 6743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:42.074632 6743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:37:42.076027 6743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:37:42.076118 6743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:42.076225 6743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:37:42.076292 6743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:42.104776 6743 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.131:40907
I20260618 22:37:42.104828 6869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.131:40907 every 8 connection(s)
I20260618 22:37:42.106079 6743 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/data/info.pb
I20260618 22:37:42.110311 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 6743
I20260618 22:37:42.110498 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/wal/instance
I20260618 22:37:42.111047 19034 external_mini_cluster.cc:1428] Setting key 5a5ceb2ff3ab6dc00815843b61a5c5aa
I20260618 22:37:42.118916 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.109301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:41741 (local address 127.18.150.190:37159)
0618 22:37:42.109479 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:42.109483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:42.111175 (+ 1692us) server_negotiation.cc:407] Connection header received
0618 22:37:42.113152 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:42.113156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:42.113220 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:42.113294 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:42.114462 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.115142 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.116596 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.116971 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.117056 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.118318 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:42.118323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:42.118326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:42.118343 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:42.118375 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:42.118387 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:42.118448 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:42.118615 (+ 167us) server_negotiation.cc:299] Negotiation successful
0618 22:37:42.118694 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:37:42.119880 6872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.109833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:37159 (local address 127.18.150.131:41741)
0618 22:37:42.110964 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.111015 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:42.112878 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:42.113527 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:42.113538 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:42.113664 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:37:42.114315 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.114333 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.115290 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.115293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.116492 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.116504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.117162 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.117168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.117395 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.118016 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:37:42.118019 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:37:42.118057 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:37:42.118138 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:37:42.118455 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:37:42.118462 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:37:42.118467 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:37:42.118591 (+ 124us) client_negotiation.cc:240] Negotiation successful
0618 22:37:42.118876 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":321,"threads_started":1}
I20260618 22:37:42.121058 6870 heartbeater.cc:347] Connected to a master server at 127.18.150.190:37159
I20260618 22:37:42.121351 6870 heartbeater.cc:464] Registering TS with master...
I20260618 22:37:42.122092 6870 heartbeater.cc:511] Master 127.18.150.190:37159 requested a full tablet report, sending...
I20260618 22:37:42.123363 6424 ts_manager.cc:194] Registered new tserver with Master: 518aee346e56405bb294aafdfb0688c2 (127.18.150.131:40907)
I20260618 22:37:42.124161 6424 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.150.131:41741
I20260618 22:37:42.126097 19034 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:37:42.135646 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.130139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57386 (local address 127.18.150.190:37159)
0618 22:37:42.130288 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:42.130292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:42.130856 (+ 564us) server_negotiation.cc:407] Connection header received
0618 22:37:42.130999 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:42.131002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:42.131083 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:42.131170 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:42.131647 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.132379 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.133588 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.134026 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.134124 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.135089 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:42.135094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:42.135098 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:42.135115 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:42.135146 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:42.135157 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:42.135223 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:42.135327 (+ 104us) server_negotiation.cc:299] Negotiation successful
0618 22:37:42.135398 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:37:42.139344 6424 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:37:42.146077 6424 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:37:42.178454 6882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.157646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41527 (local address 127.0.0.1:33748)
0618 22:37:42.158429 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.158502 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:42.158690 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:42.171739 (+ 13049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:42.171748 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:42.171783 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:42.172194 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.172206 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.173917 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.173923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.175795 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.175811 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.177575 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.177581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.177769 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.177833 (+ 64us) client_negotiation.cc:769] Sending connection context
0618 22:37:42.177962 (+ 129us) client_negotiation.cc:240] Negotiation successful
0618 22:37:42.178149 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":1127,"threads_started":1}
I20260618 22:37:42.178501 6884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.157879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.18.150.129:41527)
0618 22:37:42.170865 (+ 12986us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:42.170874 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:42.170907 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:37:42.171017 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:42.171025 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:42.171318 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:42.171549 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:42.172420 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.173783 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.176030 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.177419 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.177549 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.177770 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:42.178031 (+ 261us) server_negotiation.cc:299] Negotiation successful
0618 22:37:42.178185 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12812,"thread_start_us":13161,"threads_started":1}
I20260618 22:37:42.179984 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.159370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:41809 (local address 127.0.0.1:55766)
0618 22:37:42.172746 (+ 13376us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.172763 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:42.172885 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:42.173437 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:42.173441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:42.173464 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:42.173771 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.173783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.175536 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.175540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.177185 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.177197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.179287 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.179293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.179467 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.179534 (+ 67us) client_negotiation.cc:769] Sending connection context
0618 22:37:42.179621 (+ 87us) client_negotiation.cc:240] Negotiation successful
0618 22:37:42.179740 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13244,"mutex_wait_us":67,"thread_start_us":13100,"threads_started":1}
I20260618 22:37:42.180150 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.160064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.18.150.130:41809)
0618 22:37:42.172314 (+ 12250us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:42.172322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:42.172875 (+ 553us) server_negotiation.cc:407] Connection header received
0618 22:37:42.173021 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:42.173028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:42.173230 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:42.173430 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:42.173981 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.175384 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.177369 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.179072 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.179233 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.179427 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:42.179706 (+ 279us) server_negotiation.cc:299] Negotiation successful
0618 22:37:42.179860 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12109,"thread_start_us":12004,"threads_started":1}
I20260618 22:37:42.180791 6883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.157729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.131:40907 (local address 127.0.0.1:50806)
0618 22:37:42.171675 (+ 13946us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.171695 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:42.171812 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:42.172533 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:42.172538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:42.172569 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:42.172941 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.172951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.174945 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.174951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.177155 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.177170 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.179256 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.179267 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.179449 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.179552 (+ 103us) client_negotiation.cc:769] Sending connection context
0618 22:37:42.179636 (+ 84us) client_negotiation.cc:240] Negotiation successful
0618 22:37:42.180457 (+ 821us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13807,"mutex_wait_us":113,"thread_start_us":13665,"threads_started":1}
I20260618 22:37:42.181502 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.158462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50806 (local address 127.18.150.131:40907)
0618 22:37:42.170435 (+ 11973us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:42.170444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:42.171927 (+ 1483us) server_negotiation.cc:407] Connection header received
0618 22:37:42.172033 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:42.172040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:42.172307 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:42.173082 (+ 775us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:42.173288 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.174800 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.177359 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.178743 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.178922 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.179117 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:42.181007 (+ 1890us) server_negotiation.cc:299] Negotiation successful
0618 22:37:42.181164 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11767,"thread_start_us":13273,"threads_started":1}
I20260618 22:37:42.182600 6545 tablet_service.cc:1467] Processing CreateTablet for tablet 649bfd1d5812495d819e122d9526b508 (DEFAULT_TABLE table=test-table [id=97578a1f1dba4c1292c2982d5f15983e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:42.183388 6675 tablet_service.cc:1467] Processing CreateTablet for tablet 649bfd1d5812495d819e122d9526b508 (DEFAULT_TABLE table=test-table [id=97578a1f1dba4c1292c2982d5f15983e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:42.183943 6545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 649bfd1d5812495d819e122d9526b508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:42.184403 6675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 649bfd1d5812495d819e122d9526b508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:42.185346 6805 tablet_service.cc:1467] Processing CreateTablet for tablet 649bfd1d5812495d819e122d9526b508 (DEFAULT_TABLE table=test-table [id=97578a1f1dba4c1292c2982d5f15983e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:37:42.186652 6805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 649bfd1d5812495d819e122d9526b508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:42.195158 6888 tablet_bootstrap.cc:492] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298: Bootstrap starting.
I20260618 22:37:42.198561 6888 tablet_bootstrap.cc:654] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:42.199559 6888 log.cc:824] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:42.200163 6890 tablet_bootstrap.cc:492] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2: Bootstrap starting.
I20260618 22:37:42.201601 6889 tablet_bootstrap.cc:492] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc: Bootstrap starting.
I20260618 22:37:42.204131 6890 tablet_bootstrap.cc:654] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:42.204308 6889 tablet_bootstrap.cc:654] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:42.205263 6889 log.cc:824] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:42.205428 6890 log.cc:824] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:42.208222 6888 tablet_bootstrap.cc:492] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298: No bootstrap required, opened a new log
I20260618 22:37:42.208559 6888 ts_tablet_manager.cc:1397] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20260618 22:37:42.209142 6889 tablet_bootstrap.cc:492] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc: No bootstrap required, opened a new log
I20260618 22:37:42.209227 6890 tablet_bootstrap.cc:492] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2: No bootstrap required, opened a new log
I20260618 22:37:42.209417 6889 ts_tablet_manager.cc:1397] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260618 22:37:42.209554 6890 ts_tablet_manager.cc:1397] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260618 22:37:42.216209 6890 raft_consensus.cc:348] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51364d1b942647edb9cae3b9c09d4bcc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41527 } } peers { permanent_uuid: "518aee346e56405bb294aafdfb0688c2" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 40907 } } peers { permanent_uuid: "f008c691fc2c4b148b54b0c9f7481298" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41809 } }
I20260618 22:37:42.216565 6890 raft_consensus.cc:374] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:42.216670 6890 raft_consensus.cc:729] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 518aee346e56405bb294aafdfb0688c2, State: Initialized, Role: FOLLOWER
I20260618 22:37:42.217240 6890 consensus_queue.cc:260] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51364d1b942647edb9cae3b9c09d4bcc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41527 } } peers { permanent_uuid: "518aee346e56405bb294aafdfb0688c2" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 40907 } } peers { permanent_uuid: "f008c691fc2c4b148b54b0c9f7481298" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41809 } }
I20260618 22:37:42.218523 6870 heartbeater.cc:503] Master 127.18.150.190:37159 was elected leader, sending a full tablet report...
I20260618 22:37:42.219040 6890 ts_tablet_manager.cc:1428] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260618 22:37:42.219364 6889 raft_consensus.cc:348] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51364d1b942647edb9cae3b9c09d4bcc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41527 } } peers { permanent_uuid: "518aee346e56405bb294aafdfb0688c2" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 40907 } } peers { permanent_uuid: "f008c691fc2c4b148b54b0c9f7481298" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41809 } }
I20260618 22:37:42.219776 6889 raft_consensus.cc:374] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:42.219875 6889 raft_consensus.cc:729] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51364d1b942647edb9cae3b9c09d4bcc, State: Initialized, Role: FOLLOWER
I20260618 22:37:42.220019 6888 raft_consensus.cc:348] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51364d1b942647edb9cae3b9c09d4bcc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41527 } } peers { permanent_uuid: "518aee346e56405bb294aafdfb0688c2" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 40907 } } peers { permanent_uuid: "f008c691fc2c4b148b54b0c9f7481298" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41809 } }
I20260618 22:37:42.220455 6888 raft_consensus.cc:374] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:42.220568 6888 raft_consensus.cc:729] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f008c691fc2c4b148b54b0c9f7481298, State: Initialized, Role: FOLLOWER
I20260618 22:37:42.220454 6889 consensus_queue.cc:260] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51364d1b942647edb9cae3b9c09d4bcc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41527 } } peers { permanent_uuid: "518aee346e56405bb294aafdfb0688c2" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 40907 } } peers { permanent_uuid: "f008c691fc2c4b148b54b0c9f7481298" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41809 } }
I20260618 22:37:42.221277 6888 consensus_queue.cc:260] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51364d1b942647edb9cae3b9c09d4bcc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41527 } } peers { permanent_uuid: "518aee346e56405bb294aafdfb0688c2" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 40907 } } peers { permanent_uuid: "f008c691fc2c4b148b54b0c9f7481298" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41809 } }
I20260618 22:37:42.222425 6740 heartbeater.cc:503] Master 127.18.150.190:37159 was elected leader, sending a full tablet report...
I20260618 22:37:42.223248 6888 ts_tablet_manager.cc:1428] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260618 22:37:42.224432 6610 heartbeater.cc:503] Master 127.18.150.190:37159 was elected leader, sending a full tablet report...
I20260618 22:37:42.225021 6889 ts_tablet_manager.cc:1428] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
W20260618 22:37:42.234876 6611 tablet.cc:2369] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:37:42.243242 6894 raft_consensus.cc:482] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:37:42.243492 6894 raft_consensus.cc:504] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51364d1b942647edb9cae3b9c09d4bcc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41527 } } peers { permanent_uuid: "518aee346e56405bb294aafdfb0688c2" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 40907 } } peers { permanent_uuid: "f008c691fc2c4b148b54b0c9f7481298" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41809 } }
I20260618 22:37:42.245117 6894 leader_election.cc:288] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51364d1b942647edb9cae3b9c09d4bcc (127.18.150.129:41527), f008c691fc2c4b148b54b0c9f7481298 (127.18.150.130:41809)
I20260618 22:37:42.251201 6872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.245409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.130:41809 (local address 127.18.150.131:43005)
0618 22:37:42.245686 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.245717 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:42.245866 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:42.246382 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:42.246388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:42.246418 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:42.246618 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.246630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.247776 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.247781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.249314 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.249326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.250458 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.250464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.250639 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.250678 (+ 39us) client_negotiation.cc:769] Sending connection context
0618 22:37:42.250770 (+ 92us) client_negotiation.cc:240] Negotiation successful
0618 22:37:42.250881 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":49}
I20260618 22:37:42.251216 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.245507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:43005 (local address 127.18.150.130:41809)
0618 22:37:42.245690 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:42.245696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:42.245792 (+ 96us) server_negotiation.cc:407] Connection header received
0618 22:37:42.246048 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:42.246053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:42.246144 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:42.246252 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:42.246826 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.247625 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.249475 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.250310 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.250425 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.250475 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:42.250833 (+ 358us) server_negotiation.cc:299] Negotiation successful
0618 22:37:42.250958 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:37:42.251300 6884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.245582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.150.131:34919 (local address 127.18.150.129:41527)
0618 22:37:42.245743 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:42.245747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:42.246306 (+ 559us) server_negotiation.cc:407] Connection header received
0618 22:37:42.246449 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:42.246455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:42.246541 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:42.246643 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:37:42.247189 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.247931 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.249645 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.250501 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.250642 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.250707 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:42.250935 (+ 228us) server_negotiation.cc:299] Negotiation successful
0618 22:37:42.251062 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:37:42.251283 6897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.245507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41527 (local address 127.18.150.131:34919)
0618 22:37:42.246188 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.246210 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:37:42.246331 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:37:42.246657 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:37:42.246661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:37:42.246679 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:37:42.247023 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.247035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.248074 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.248077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.249445 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:37:42.249458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.250662 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.250667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:37:42.250785 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.250815 (+ 30us) client_negotiation.cc:769] Sending connection context
0618 22:37:42.250898 (+ 83us) client_negotiation.cc:240] Negotiation successful
0618 22:37:42.251015 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":347,"threads_started":1}
I20260618 22:37:42.252408 6695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "649bfd1d5812495d819e122d9526b508" candidate_uuid: "518aee346e56405bb294aafdfb0688c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f008c691fc2c4b148b54b0c9f7481298" is_pre_election: true
I20260618 22:37:42.252559 6565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "649bfd1d5812495d819e122d9526b508" candidate_uuid: "518aee346e56405bb294aafdfb0688c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51364d1b942647edb9cae3b9c09d4bcc" is_pre_election: true
I20260618 22:37:42.253027 6565 raft_consensus.cc:2454] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 518aee346e56405bb294aafdfb0688c2 in term 0.
I20260618 22:37:42.253028 6695 raft_consensus.cc:2454] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 518aee346e56405bb294aafdfb0688c2 in term 0.
I20260618 22:37:42.253656 6759 leader_election.cc:302] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51364d1b942647edb9cae3b9c09d4bcc, 518aee346e56405bb294aafdfb0688c2; no voters:
I20260618 22:37:42.254036 6894 raft_consensus.cc:2781] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:37:42.254187 6894 raft_consensus.cc:482] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:37:42.254285 6894 raft_consensus.cc:3037] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:42.256587 6894 raft_consensus.cc:504] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51364d1b942647edb9cae3b9c09d4bcc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41527 } } peers { permanent_uuid: "518aee346e56405bb294aafdfb0688c2" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 40907 } } peers { permanent_uuid: "f008c691fc2c4b148b54b0c9f7481298" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41809 } }
I20260618 22:37:42.257234 6894 leader_election.cc:288] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [CANDIDATE]: Term 1 election: Requested vote from peers 51364d1b942647edb9cae3b9c09d4bcc (127.18.150.129:41527), f008c691fc2c4b148b54b0c9f7481298 (127.18.150.130:41809)
I20260618 22:37:42.257606 6565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "649bfd1d5812495d819e122d9526b508" candidate_uuid: "518aee346e56405bb294aafdfb0688c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51364d1b942647edb9cae3b9c09d4bcc"
I20260618 22:37:42.257786 6565 raft_consensus.cc:3037] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:42.257898 6695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "649bfd1d5812495d819e122d9526b508" candidate_uuid: "518aee346e56405bb294aafdfb0688c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f008c691fc2c4b148b54b0c9f7481298"
I20260618 22:37:42.258109 6695 raft_consensus.cc:3037] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:42.260872 6565 raft_consensus.cc:2454] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 518aee346e56405bb294aafdfb0688c2 in term 1.
I20260618 22:37:42.261085 6695 raft_consensus.cc:2454] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 518aee346e56405bb294aafdfb0688c2 in term 1.
I20260618 22:37:42.261241 6759 leader_election.cc:302] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51364d1b942647edb9cae3b9c09d4bcc, 518aee346e56405bb294aafdfb0688c2; no voters:
I20260618 22:37:42.261524 6894 raft_consensus.cc:2781] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:42.261979 6894 raft_consensus.cc:686] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 1 LEADER]: Becoming Leader. State: Replica: 518aee346e56405bb294aafdfb0688c2, State: Running, Role: LEADER
I20260618 22:37:42.262476 6894 consensus_queue.cc:237] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51364d1b942647edb9cae3b9c09d4bcc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41527 } } peers { permanent_uuid: "518aee346e56405bb294aafdfb0688c2" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 40907 } } peers { permanent_uuid: "f008c691fc2c4b148b54b0c9f7481298" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41809 } }
I20260618 22:37:42.266837 6423 catalog_manager.cc:5495] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 518aee346e56405bb294aafdfb0688c2 (127.18.150.131). New cstate: current_term: 1 leader_uuid: "518aee346e56405bb294aafdfb0688c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51364d1b942647edb9cae3b9c09d4bcc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "518aee346e56405bb294aafdfb0688c2" member_type: VOTER last_known_addr { host: "127.18.150.131" port: 40907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f008c691fc2c4b148b54b0c9f7481298" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41809 } health_report { overall_health: UNKNOWN } } }
I20260618 22:37:42.292709 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.287685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50822 (local address 127.18.150.131:40907)
0618 22:37:42.287905 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:42.287910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:42.287926 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:37:42.288115 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:42.288118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:42.288199 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:42.288320 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:37:42.288586 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.289320 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.290702 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.291092 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.291181 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.291633 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:37:42.292196 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:37:42.292302 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:42.292415 (+ 113us) server_negotiation.cc:299] Negotiation successful
0618 22:37:42.292473 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:37:42.300403 6565 raft_consensus.cc:1261] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc [term 1 FOLLOWER]: Refusing update from remote peer 518aee346e56405bb294aafdfb0688c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:42.300403 6695 raft_consensus.cc:1261] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298 [term 1 FOLLOWER]: Refusing update from remote peer 518aee346e56405bb294aafdfb0688c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:37:42.301412 6898 consensus_queue.cc:1035] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f008c691fc2c4b148b54b0c9f7481298" member_type: VOTER last_known_addr { host: "127.18.150.130" port: 41809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:37:42.301726 6894 consensus_queue.cc:1035] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51364d1b942647edb9cae3b9c09d4bcc" member_type: VOTER last_known_addr { host: "127.18.150.129" port: 41527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:37:42.326813 6423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:42.327093 6423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:37:42.330447 6423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4852883bf524b3fbad9dcdc5cbbd445: Sending DeleteTablet for 3 replicas of tablet 649bfd1d5812495d819e122d9526b508
I20260618 22:37:42.331463 6805 tablet_service.cc:1514] Processing DeleteTablet for tablet 649bfd1d5812495d819e122d9526b508 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:42 UTC) from {username='slave'} at 127.0.0.1:50806
I20260618 22:37:42.331712 6675 tablet_service.cc:1514] Processing DeleteTablet for tablet 649bfd1d5812495d819e122d9526b508 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:42 UTC) from {username='slave'} at 127.0.0.1:55766
I20260618 22:37:42.332096 6545 tablet_service.cc:1514] Processing DeleteTablet for tablet 649bfd1d5812495d819e122d9526b508 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:37:42 UTC) from {username='slave'} at 127.0.0.1:33748
I20260618 22:37:42.332533 6912 tablet_replica.cc:330] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298: stopping tablet replica
I20260618 22:37:42.333021 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 6483
I20260618 22:37:42.333190 6912 raft_consensus.cc:2229] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:37:42.333966 6912 raft_consensus.cc:2258] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:42.336186 6912 ts_tablet_manager.cc:1905] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:42.343456 6912 ts_tablet_manager.cc:1918] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:42.343709 6912 log.cc:1196] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-1/wal/wals/649bfd1d5812495d819e122d9526b508
I20260618 22:37:42.344290 6912 ts_tablet_manager.cc:1939] T 649bfd1d5812495d819e122d9526b508 P f008c691fc2c4b148b54b0c9f7481298: Deleting consensus metadata
I20260618 22:37:42.346269 6411 catalog_manager.cc:4841] TS f008c691fc2c4b148b54b0c9f7481298 (127.18.150.130:41809): tablet 649bfd1d5812495d819e122d9526b508 (table test-table [id=97578a1f1dba4c1292c2982d5f15983e]) successfully deleted
I20260618 22:37:42.350888 6913 tablet_replica.cc:330] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc: stopping tablet replica
I20260618 22:37:42.351521 6913 raft_consensus.cc:2229] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:37:42.352239 6911 tablet_replica.cc:330] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2: stopping tablet replica
I20260618 22:37:42.352381 6913 raft_consensus.cc:2258] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:42.352739 6911 raft_consensus.cc:2229] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:37:42.353077 6884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.335356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33764 (local address 127.18.150.129:41527)
0618 22:37:42.335521 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:42.335529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:42.344374 (+ 8845us) server_negotiation.cc:407] Connection header received
0618 22:37:42.344867 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:42.344874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:42.345013 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:42.345094 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:42.345834 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.347161 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.349018 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.349689 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.349816 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.351182 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:42.351195 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:42.351223 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:42.352146 (+ 923us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:42.352227 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:42.352254 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:42.352364 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:42.352580 (+ 216us) server_negotiation.cc:299] Negotiation successful
0618 22:37:42.352690 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:37:42.353994 6911 raft_consensus.cc:2258] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:37:42.354391 6605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:42.359704 6911 ts_tablet_manager.cc:1905] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:42.364481 6911 ts_tablet_manager.cc:1918] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:37:42.364625 6911 log.cc:1196] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2: Deleting WAL directory at /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821960891835-19034-0/minicluster-data/ts-2/wal/wals/649bfd1d5812495d819e122d9526b508
I20260618 22:37:42.365057 6911 ts_tablet_manager.cc:1939] T 649bfd1d5812495d819e122d9526b508 P 518aee346e56405bb294aafdfb0688c2: Deleting consensus metadata
I20260618 22:37:42.366451 6408 catalog_manager.cc:4841] TS 518aee346e56405bb294aafdfb0688c2 (127.18.150.131:40907): tablet 649bfd1d5812495d819e122d9526b508 (table test-table [id=97578a1f1dba4c1292c2982d5f15983e]) successfully deleted
I20260618 22:37:42.354458 6913 ts_tablet_manager.cc:1905] T 649bfd1d5812495d819e122d9526b508 P 51364d1b942647edb9cae3b9c09d4bcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:37:42.504628 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 6483
I20260618 22:37:42.519348 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 6613
W20260618 22:37:42.519407 6409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.150.129:41527 (error 108)
I20260618 22:37:42.521337 6883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.520863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41527 (local address 127.0.0.1:33778)
0618 22:37:42.521032 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.521151 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260618 22:37:42.522533 6409 catalog_manager.cc:4584] TS 51364d1b942647edb9cae3b9c09d4bcc (127.18.150.129:41527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 649bfd1d5812495d819e122d9526b508: Network error: Client connection negotiation failed: client connection to 127.18.150.129:41527: connect: Connection refused (error 111)
I20260618 22:37:42.527174 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.520535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55778 (local address 127.18.150.130:41809)
0618 22:37:42.520704 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:42.520710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:42.520729 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:37:42.520795 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:42.520812 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:42.520929 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:42.521033 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:42.521531 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.522331 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.524118 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.524519 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.524612 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.525747 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:42.525756 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:42.525770 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:42.526449 (+ 679us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:42.526500 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:42.526526 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:42.526616 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:42.526771 (+ 155us) server_negotiation.cc:299] Negotiation successful
0618 22:37:42.526864 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:37:42.527839 6735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:42.576275 6883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.575726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41527 (local address 127.0.0.1:33790)
0618 22:37:42.575973 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.576078 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260618 22:37:42.577404 6883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.576997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41527 (local address 127.0.0.1:33794)
0618 22:37:42.577150 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.577258 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260618 22:37:42.616129 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 6613
I20260618 22:37:42.630156 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 6743
I20260618 22:37:42.638043 6883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.637501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41527 (local address 127.0.0.1:33800)
0618 22:37:42.637701 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.637813 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260618 22:37:42.638501 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.630906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50826 (local address 127.18.150.131:40907)
0618 22:37:42.631077 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:42.631084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:42.631284 (+ 200us) server_negotiation.cc:407] Connection header received
0618 22:37:42.631473 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:42.631477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:42.631554 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:42.631631 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:42.632067 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.632856 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.634583 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:42.634996 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:42.635089 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:42.636166 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:42.636175 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:42.636189 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:42.636798 (+ 609us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:42.636846 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:42.636866 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:42.636942 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:42.637674 (+ 732us) server_negotiation.cc:299] Negotiation successful
0618 22:37:42.637758 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:37:42.639112 6883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.638743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41527 (local address 127.0.0.1:33808)
0618 22:37:42.638885 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.638967 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260618 22:37:42.639299 6865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:42.720160 6883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.719617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41527 (local address 127.0.0.1:33814)
0618 22:37:42.719851 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.719993 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260618 22:37:42.721374 6883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:42.720911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.129:41527 (local address 127.0.0.1:33820)
0618 22:37:42.721136 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0618 22:37:42.721224 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.129:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260618 22:37:42.759577 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 6743
I20260618 22:37:42.775334 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 6393
I20260618 22:37:42.776464 6454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:42.884951 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 6393
2026-06-18T22:37:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2628 ms)
[----------] 6 tests from EncryptionPolicyTest (20157 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:42 dist-test-slave-h7jz krb5kdc[6922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:42 dist-test-slave-h7jz krb5kdc[6922](info): set up 2 sockets
Jun 18 22:37:42 dist-test-slave-h7jz krb5kdc[6922](info): commencing operation
krb5kdc: starting...
W20260618 22:37:43.910161 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.010s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:44 dist-test-slave-h7jz krb5kdc[6922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:37:44.141095 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:44.456748 6935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:44.457050 6935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:44.457188 6935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:44.464389 6935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:44.464514 6935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:44.464593 6935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:44.464679 6935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:44.464737 6935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:44.474923 6935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46641
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:44.476191 6935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:44.477453 6935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:44.488646 6940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:44.488708 6943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:44.488729 6941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:44.490000 6935 server_base.cc:1034] running on GCE node
I20260618 22:37:44.490712 6935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:44.492093 6935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:44.493287 6935 hybrid_clock.cc:648] HybridClock initialized: now 1781822264493239 us; error 38 us; skew 500 ppm
I20260618 22:37:44.493614 6935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:44 dist-test-slave-h7jz krb5kdc[6922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822264, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:44.496464 6935 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:37:44.497831 6935 webserver.cc:460] Webserver started at http://127.0.0.1:43505/ using document root <none> and password file <none>
I20260618 22:37:44.498519 6935 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:44.498631 6935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:44.498911 6935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:44.501869 6935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "d37447aae0d74f7e9c8b362ce1e227cb"
format_stamp: "Formatted at 2026-06-18 22:37:44 on dist-test-slave-h7jz"
server_key: "72de70f814e5e6939e365630a5a89e5e"
server_key_iv: "d1111cb5f022091a48ee3c14083d0d23"
server_key_version: "encryptionkey@0"
I20260618 22:37:44.502825 6935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "d37447aae0d74f7e9c8b362ce1e227cb"
format_stamp: "Formatted at 2026-06-18 22:37:44 on dist-test-slave-h7jz"
server_key: "72de70f814e5e6939e365630a5a89e5e"
server_key_iv: "d1111cb5f022091a48ee3c14083d0d23"
server_key_version: "encryptionkey@0"
I20260618 22:37:44.507984 6935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260618 22:37:44.511348 6950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:44.512634 6935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:37:44.512799 6935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "d37447aae0d74f7e9c8b362ce1e227cb"
format_stamp: "Formatted at 2026-06-18 22:37:44 on dist-test-slave-h7jz"
server_key: "72de70f814e5e6939e365630a5a89e5e"
server_key_iv: "d1111cb5f022091a48ee3c14083d0d23"
server_key_version: "encryptionkey@0"
I20260618 22:37:44.512972 6935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:44.529040 6935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:44.533035 6935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:44.533324 6935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:44.550539 6935 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46641
I20260618 22:37:44.550555 7001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46641 every 8 connection(s)
I20260618 22:37:44.551878 6935 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:44.553574 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 6935
I20260618 22:37:44.553790 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:44.554586 19034 external_mini_cluster.cc:1428] Setting key 58f45ad23ecfccb9b41c7c1a8f82b474
I20260618 22:37:44.556356 7002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 18 22:37:44 dist-test-slave-h7jz krb5kdc[6922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:37:44.570297 7002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb: Bootstrap starting.
I20260618 22:37:44.573642 7002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:44.574785 7002 log.cc:824] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:44.577878 7002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb: No bootstrap required, opened a new log
I20260618 22:37:44.578720 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:44.556519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51609 (local address 127.0.0.1:46641)
0618 22:37:44.557271 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:44.557287 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:44.557338 (+ 51us) server_negotiation.cc:407] Connection header received
0618 22:37:44.559382 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:44.559411 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:44.559986 (+ 575us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:44.560424 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:44.561685 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:44.563667 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:44.564917 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:44.565905 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:44.566162 (+ 257us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:44.571157 (+ 4995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:44.571165 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:44.571180 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:44.571243 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:44.574316 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:44.575288 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:44.575298 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:44.575306 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:44.575543 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:44.575993 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:44.575999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:44.576002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:44.576528 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:44.576761 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:44.577283 (+ 522us) server_negotiation.cc:299] Negotiation successful
0618 22:37:44.577565 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":303,"threads_started":1}
I20260618 22:37:44.582629 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:37:44.586920 7002 raft_consensus.cc:348] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37447aae0d74f7e9c8b362ce1e227cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } }
I20260618 22:37:44.587422 7002 raft_consensus.cc:374] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:44.587532 7002 raft_consensus.cc:729] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37447aae0d74f7e9c8b362ce1e227cb, State: Initialized, Role: FOLLOWER
I20260618 22:37:44.588179 7002 consensus_queue.cc:260] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37447aae0d74f7e9c8b362ce1e227cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } }
I20260618 22:37:44.588425 7002 raft_consensus.cc:388] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:44.588610 7002 raft_consensus.cc:482] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:44.588781 7002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:44.591704 7002 raft_consensus.cc:504] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37447aae0d74f7e9c8b362ce1e227cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } }
I20260618 22:37:44.592200 7002 leader_election.cc:302] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d37447aae0d74f7e9c8b362ce1e227cb; no voters:
I20260618 22:37:44.592795 7002 leader_election.cc:288] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:44.592901 7012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:44.594107 7012 raft_consensus.cc:686] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [term 1 LEADER]: Becoming Leader. State: Replica: d37447aae0d74f7e9c8b362ce1e227cb, State: Running, Role: LEADER
I20260618 22:37:44.594712 7012 consensus_queue.cc:237] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37447aae0d74f7e9c8b362ce1e227cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } }
I20260618 22:37:44.595155 7002 sys_catalog.cc:564] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:44.597520 7013 sys_catalog.cc:455] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d37447aae0d74f7e9c8b362ce1e227cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37447aae0d74f7e9c8b362ce1e227cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } } }
I20260618 22:37:44.597913 7013 sys_catalog.cc:458] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:44.599185 7014 sys_catalog.cc:455] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d37447aae0d74f7e9c8b362ce1e227cb. Latest consensus state: current_term: 1 leader_uuid: "d37447aae0d74f7e9c8b362ce1e227cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37447aae0d74f7e9c8b362ce1e227cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46641 } } }
I20260618 22:37:44.599550 7014 sys_catalog.cc:458] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:44.603521 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:44.588350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42191 (local address 127.0.0.1:46641)
0618 22:37:44.588547 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:44.588554 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:44.588992 (+ 438us) server_negotiation.cc:407] Connection header received
0618 22:37:44.589170 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:44.589173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:44.589240 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:44.589318 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:44.591060 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:44.591803 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:44.593148 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:44.593765 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:44.593881 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:44.596560 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:44.596565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:44.596570 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:44.596611 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:44.600248 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:44.601166 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:44.601172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:44.601176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:44.601402 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:44.601858 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:44.601863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:44.601867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:44.602312 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:44.602655 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:44.602973 (+ 318us) server_negotiation.cc:299] Negotiation successful
0618 22:37:44.603167 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:37:44.606666 7017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:44.612226 7017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:44.620560 7017 catalog_manager.cc:1347] Generated new cluster ID: 835d0af0e0274e6fae3ab3a9d1aa7644
I20260618 22:37:44.620726 7017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:44.639894 7017 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:44.641209 7017 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:44.649106 7017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d37447aae0d74f7e9c8b362ce1e227cb: Generated new TSK 0
I20260618 22:37:44.649920 7017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:44.659679 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 6935
I20260618 22:37:44.661353 6997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:44.752367 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 6935
W20260618 22:37:44.766028 6920 connection.cc:535] client connection to 127.0.0.1:46641 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46641): Connection reset by peer (error 104)
2026-06-18T22:37:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1863 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:44 dist-test-slave-h7jz krb5kdc[7034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:44 dist-test-slave-h7jz krb5kdc[7034](info): set up 2 sockets
Jun 18 22:37:44 dist-test-slave-h7jz krb5kdc[7034](info): commencing operation
krb5kdc: starting...
W20260618 22:37:45.773998 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:45 dist-test-slave-h7jz krb5kdc[7034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:37:46.011016 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33493
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:46.325685 7047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:46.325999 7047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:46.326128 7047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:46.333312 7047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:46.333422 7047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:46.333494 7047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:46.333580 7047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:46.333637 7047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:46.343567 7047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33493
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:46.344846 7047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:46.346278 7047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:46.357600 7053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:46.357574 7052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:46.357555 7055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:46.358376 7047 server_base.cc:1034] running on GCE node
I20260618 22:37:46.359117 7047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:46.360487 7047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:46.361697 7047 hybrid_clock.cc:648] HybridClock initialized: now 1781822266361657 us; error 30 us; skew 500 ppm
I20260618 22:37:46.362077 7047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:46 dist-test-slave-h7jz krb5kdc[7034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822266, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:46.365374 7047 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:37:46.366844 7047 webserver.cc:460] Webserver started at http://127.0.0.1:45211/ using document root <none> and password file <none>
I20260618 22:37:46.367527 7047 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:46.367635 7047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:46.367924 7047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:46.370978 7047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "8fc8434c415d4c90a98366ab093b877e"
format_stamp: "Formatted at 2026-06-18 22:37:46 on dist-test-slave-h7jz"
server_key: "b74423a1bdf82bb03db946eb6af2769e"
server_key_iv: "496314a703806933ae4929e83ab96c18"
server_key_version: "encryptionkey@0"
I20260618 22:37:46.371737 7047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "8fc8434c415d4c90a98366ab093b877e"
format_stamp: "Formatted at 2026-06-18 22:37:46 on dist-test-slave-h7jz"
server_key: "b74423a1bdf82bb03db946eb6af2769e"
server_key_iv: "496314a703806933ae4929e83ab96c18"
server_key_version: "encryptionkey@0"
I20260618 22:37:46.376817 7047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:37:46.380287 7062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:46.381668 7047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:37:46.381831 7047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "8fc8434c415d4c90a98366ab093b877e"
format_stamp: "Formatted at 2026-06-18 22:37:46 on dist-test-slave-h7jz"
server_key: "b74423a1bdf82bb03db946eb6af2769e"
server_key_iv: "496314a703806933ae4929e83ab96c18"
server_key_version: "encryptionkey@0"
I20260618 22:37:46.382054 7047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:46.406459 7047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:46.410619 7047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:46.410873 7047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:46.427737 7047 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33493
I20260618 22:37:46.427757 7113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33493 every 8 connection(s)
I20260618 22:37:46.429064 7047 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:46.432628 7114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:46.433351 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 7047
I20260618 22:37:46.433547 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:46.434161 19034 external_mini_cluster.cc:1428] Setting key 9d6e098b97d2019a17936cc140d85cb4
I20260618 22:37:46.446377 7114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e: Bootstrap starting.
I20260618 22:37:46.448068 7117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:46.435994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38037 (local address 127.0.0.1:33493)
0618 22:37:46.436773 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:46.436788 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:46.436830 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:37:46.438864 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:46.438890 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:46.439337 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:46.439695 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:46.440308 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:46.441588 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:46.443461 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:46.444183 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:46.444335 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:46.445709 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:46.445717 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:46.445732 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:46.446340 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:46.446427 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:46.446460 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:46.446624 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:46.446908 (+ 284us) server_negotiation.cc:299] Negotiation successful
0618 22:37:46.447074 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":356,"threads_started":1}
I20260618 22:37:46.449522 7114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:46.450608 7114 log.cc:824] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:46.451690 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:37:46.453188 7114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e: No bootstrap required, opened a new log
I20260618 22:37:46.460892 7114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc8434c415d4c90a98366ab093b877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33493 } }
I20260618 22:37:46.461264 7114 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:46.461369 7114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fc8434c415d4c90a98366ab093b877e, State: Initialized, Role: FOLLOWER
I20260618 22:37:46.461854 7117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:46.455766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45657 (local address 127.0.0.1:33493)
0618 22:37:46.455911 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:46.455915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:46.456401 (+ 486us) server_negotiation.cc:407] Connection header received
0618 22:37:46.456572 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:46.456576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:46.456635 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:46.456737 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:46.457216 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:46.458069 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:46.459370 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:46.459850 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:46.459969 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:46.461096 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:46.461101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:46.461104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:46.461124 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:46.461177 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:46.461193 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:46.461293 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:46.461501 (+ 208us) server_negotiation.cc:299] Negotiation successful
0618 22:37:46.461554 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:37:46.462045 7114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc8434c415d4c90a98366ab093b877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33493 } }
I20260618 22:37:46.462293 7114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:46.462433 7114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:46.462597 7114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:46.464824 7114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc8434c415d4c90a98366ab093b877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33493 } }
I20260618 22:37:46.465296 7114 leader_election.cc:302] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fc8434c415d4c90a98366ab093b877e; no voters:
I20260618 22:37:46.465819 7114 leader_election.cc:288] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:46.465996 7124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:46.466698 7124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [term 1 LEADER]: Becoming Leader. State: Replica: 8fc8434c415d4c90a98366ab093b877e, State: Running, Role: LEADER
I20260618 22:37:46.467182 7124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc8434c415d4c90a98366ab093b877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33493 } }
I20260618 22:37:46.467873 7114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:46.475385 7126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fc8434c415d4c90a98366ab093b877e. Latest consensus state: current_term: 1 leader_uuid: "8fc8434c415d4c90a98366ab093b877e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc8434c415d4c90a98366ab093b877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33493 } } }
I20260618 22:37:46.475803 7125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fc8434c415d4c90a98366ab093b877e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc8434c415d4c90a98366ab093b877e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33493 } } }
I20260618 22:37:46.476039 7125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:46.476363 7126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:46.478570 7138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:46.482659 7138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:46.490741 7138 catalog_manager.cc:1347] Generated new cluster ID: 05d025bba35f445899bafb0f8aa284d0
I20260618 22:37:46.490851 7138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:46.505662 7138 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:46.507005 7138 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:46.520007 7138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fc8434c415d4c90a98366ab093b877e: Generated new TSK 0
I20260618 22:37:46.521121 7138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:46.532482 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7047
W20260618 22:37:46.533649 7109 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38037
W20260618 22:37:46.534505 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7047: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:37:46.534705 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 7047
W20260618 22:37:46.548395 7032 connection.cc:535] client connection to 127.0.0.1:33493 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33493): Connection reset by peer (error 104)
2026-06-18T22:37:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1782 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:46 dist-test-slave-h7jz krb5kdc[7145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:46 dist-test-slave-h7jz krb5kdc[7145](info): set up 2 sockets
Jun 18 22:37:46 dist-test-slave-h7jz krb5kdc[7145](info): commencing operation
krb5kdc: starting...
W20260618 22:37:47.557917 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:47 dist-test-slave-h7jz krb5kdc[7145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:37:47.797128 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:48.117230 7158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:48.117527 7158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:48.117617 7158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:48.125131 7158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:48.125267 7158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:48.125313 7158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:48.125380 7158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:48.125423 7158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:48.135843 7158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36835
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:48.137099 7158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:48.138415 7158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:48.150118 7166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:48.150074 7163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:48.150209 7164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:48.150851 7158 server_base.cc:1034] running on GCE node
I20260618 22:37:48.151651 7158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:48.153064 7158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:48.154281 7158 hybrid_clock.cc:648] HybridClock initialized: now 1781822268154221 us; error 56 us; skew 500 ppm
I20260618 22:37:48.154606 7158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:48 dist-test-slave-h7jz krb5kdc[7145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822268, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:48.161876 7158 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:37:48.163520 7158 webserver.cc:460] Webserver started at http://127.0.0.1:39477/ using document root <none> and password file <none>
I20260618 22:37:48.164186 7158 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:48.164288 7158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:48.164538 7158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:48.167685 7158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "cefe630c5375472bbbad6be58d30a71f"
format_stamp: "Formatted at 2026-06-18 22:37:48 on dist-test-slave-h7jz"
server_key: "71da419b7a5ee04298e9b68920b83448"
server_key_iv: "757cb272077552930e11260b8f9ce74a"
server_key_version: "encryptionkey@0"
I20260618 22:37:48.168426 7158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "cefe630c5375472bbbad6be58d30a71f"
format_stamp: "Formatted at 2026-06-18 22:37:48 on dist-test-slave-h7jz"
server_key: "71da419b7a5ee04298e9b68920b83448"
server_key_iv: "757cb272077552930e11260b8f9ce74a"
server_key_version: "encryptionkey@0"
I20260618 22:37:48.173452 7158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:37:48.176808 7173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:48.178143 7158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:37:48.178274 7158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "cefe630c5375472bbbad6be58d30a71f"
format_stamp: "Formatted at 2026-06-18 22:37:48 on dist-test-slave-h7jz"
server_key: "71da419b7a5ee04298e9b68920b83448"
server_key_iv: "757cb272077552930e11260b8f9ce74a"
server_key_version: "encryptionkey@0"
I20260618 22:37:48.178444 7158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:48.183779 7158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:48.187688 7158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:48.187922 7158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:48.204480 7158 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36835
I20260618 22:37:48.204520 7224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36835 every 8 connection(s)
I20260618 22:37:48.205852 7158 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:48.209669 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 7158
I20260618 22:37:48.209842 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:48.209837 7225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:48.210414 19034 external_mini_cluster.cc:1428] Setting key 5bf06bb15074ca68b2c39ca30a921e62
I20260618 22:37:48.218037 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:48.212256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58075 (local address 127.0.0.1:36835)
0618 22:37:48.213060 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:48.213071 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:48.213114 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:37:48.214709 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:48.214732 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:48.215096 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:48.215461 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:48.216450 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:48.216459 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:48.216471 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:48.216485 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:48.216544 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:48.216567 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:48.216784 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:48.216943 (+ 159us) server_negotiation.cc:299] Negotiation successful
0618 22:37:48.217113 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":349,"threads_started":1}
I20260618 22:37:48.221863 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:37:48.223383 7225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f: Bootstrap starting.
I20260618 22:37:48.226547 7225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:48.227828 7225 log.cc:824] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:48.228371 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:48.225991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47263 (local address 127.0.0.1:36835)
0618 22:37:48.226161 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:48.226167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:48.226585 (+ 418us) server_negotiation.cc:407] Connection header received
0618 22:37:48.226729 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:48.226734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:48.226800 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:48.226919 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:48.227750 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:48.227755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:48.227760 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:48.227778 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:48.227830 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:48.227852 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:48.227968 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:48.228046 (+ 78us) server_negotiation.cc:299] Negotiation successful
0618 22:37:48.228101 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:37:48.230767 7225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f: No bootstrap required, opened a new log
I20260618 22:37:48.237793 7225 raft_consensus.cc:348] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefe630c5375472bbbad6be58d30a71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } }
I20260618 22:37:48.238221 7225 raft_consensus.cc:374] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:48.238300 7225 raft_consensus.cc:729] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cefe630c5375472bbbad6be58d30a71f, State: Initialized, Role: FOLLOWER
I20260618 22:37:48.238801 7225 consensus_queue.cc:260] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefe630c5375472bbbad6be58d30a71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } }
I20260618 22:37:48.238994 7225 raft_consensus.cc:388] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:48.239096 7225 raft_consensus.cc:482] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:48.239248 7225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:48.241458 7225 raft_consensus.cc:504] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefe630c5375472bbbad6be58d30a71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } }
I20260618 22:37:48.241925 7225 leader_election.cc:302] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cefe630c5375472bbbad6be58d30a71f; no voters:
I20260618 22:37:48.242511 7225 leader_election.cc:288] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:48.242635 7235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:48.243325 7235 raft_consensus.cc:686] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [term 1 LEADER]: Becoming Leader. State: Replica: cefe630c5375472bbbad6be58d30a71f, State: Running, Role: LEADER
I20260618 22:37:48.243892 7235 consensus_queue.cc:237] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefe630c5375472bbbad6be58d30a71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } }
I20260618 22:37:48.244465 7225 sys_catalog.cc:564] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:48.246552 7237 sys_catalog.cc:455] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cefe630c5375472bbbad6be58d30a71f. Latest consensus state: current_term: 1 leader_uuid: "cefe630c5375472bbbad6be58d30a71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefe630c5375472bbbad6be58d30a71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } } }
I20260618 22:37:48.246523 7236 sys_catalog.cc:455] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cefe630c5375472bbbad6be58d30a71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefe630c5375472bbbad6be58d30a71f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36835 } } }
I20260618 22:37:48.246942 7236 sys_catalog.cc:458] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:48.246942 7237 sys_catalog.cc:458] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:48.249508 7242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:48.254976 7242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:48.263733 7242 catalog_manager.cc:1347] Generated new cluster ID: 311e0de11ff94388a7b45870b13c8272
I20260618 22:37:48.263840 7242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:48.278306 7242 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:48.278790 7242 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:48.290813 7242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cefe630c5375472bbbad6be58d30a71f: Generated new TSK 0
I20260618 22:37:48.291770 7242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:48.297700 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7158
W20260618 22:37:48.298952 7220 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58075
W20260618 22:37:48.299713 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7158: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:37:48.299867 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 7158
2026-06-18T22:37:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1766 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:48 dist-test-slave-h7jz krb5kdc[7256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:48 dist-test-slave-h7jz krb5kdc[7256](info): set up 2 sockets
Jun 18 22:37:48 dist-test-slave-h7jz krb5kdc[7256](info): commencing operation
krb5kdc: starting...
W20260618 22:37:49.314309 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:49 dist-test-slave-h7jz krb5kdc[7256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:37:49.553081 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35537
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:49.881052 7269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:49.881372 7269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:49.881522 7269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:49.888721 7269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:49.888840 7269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:49.888921 7269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:49.889005 7269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:49.889065 7269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:49.898837 7269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:49.900034 7269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:49.901302 7269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:49.912248 7274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:49.912267 7277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:49.912200 7275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:49.912967 7269 server_base.cc:1034] running on GCE node
I20260618 22:37:49.913707 7269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:49.915100 7269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:49.916334 7269 hybrid_clock.cc:648] HybridClock initialized: now 1781822269916284 us; error 37 us; skew 500 ppm
I20260618 22:37:49.916673 7269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:49 dist-test-slave-h7jz krb5kdc[7256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822269, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:49.919593 7269 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:37:49.920928 7269 webserver.cc:460] Webserver started at http://127.0.0.1:46471/ using document root <none> and password file <none>
I20260618 22:37:49.921572 7269 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:49.921684 7269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:49.922008 7269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:49.924826 7269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "ce9f3282c784469a8d6b083650afc8e4"
format_stamp: "Formatted at 2026-06-18 22:37:49 on dist-test-slave-h7jz"
server_key: "856a5b8011f375ef2283cd7e0f007692"
server_key_iv: "5c4d492797763131f54af2d40b48d95d"
server_key_version: "encryptionkey@0"
I20260618 22:37:49.925559 7269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "ce9f3282c784469a8d6b083650afc8e4"
format_stamp: "Formatted at 2026-06-18 22:37:49 on dist-test-slave-h7jz"
server_key: "856a5b8011f375ef2283cd7e0f007692"
server_key_iv: "5c4d492797763131f54af2d40b48d95d"
server_key_version: "encryptionkey@0"
I20260618 22:37:49.930410 7269 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260618 22:37:49.933588 7284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:49.934885 7269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:37:49.935035 7269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "ce9f3282c784469a8d6b083650afc8e4"
format_stamp: "Formatted at 2026-06-18 22:37:49 on dist-test-slave-h7jz"
server_key: "856a5b8011f375ef2283cd7e0f007692"
server_key_iv: "5c4d492797763131f54af2d40b48d95d"
server_key_version: "encryptionkey@0"
I20260618 22:37:49.935204 7269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:49.956660 7269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:49.960536 7269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:49.960791 7269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:49.977811 7269 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35537
I20260618 22:37:49.977866 7335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35537 every 8 connection(s)
I20260618 22:37:49.979209 7269 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:49.982873 7336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:49.988371 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 7269
I20260618 22:37:49.988543 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:49.989025 19034 external_mini_cluster.cc:1428] Setting key af4071aa3bd95fc508a9e754252a5cb8
I20260618 22:37:49.994138 7336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4: Bootstrap starting.
I20260618 22:37:49.997874 7336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:49.999148 7336 log.cc:824] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:50.002288 7336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4: No bootstrap required, opened a new log
Jun 18 22:37:50 dist-test-slave-h7jz krb5kdc[7256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:37:50.011636 7336 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9f3282c784469a8d6b083650afc8e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35537 } }
I20260618 22:37:50.012104 7336 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:50.012207 7336 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9f3282c784469a8d6b083650afc8e4, State: Initialized, Role: FOLLOWER
I20260618 22:37:50.012897 7336 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9f3282c784469a8d6b083650afc8e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35537 } }
I20260618 22:37:50.013171 7336 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:50.013302 7336 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:50.013481 7336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:50.014456 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:49.991088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53815 (local address 127.0.0.1:35537)
0618 22:37:49.992078 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:49.992095 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:49.992446 (+ 351us) server_negotiation.cc:407] Connection header received
0618 22:37:49.994804 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:49.994832 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:49.995339 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:49.995777 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:49.996997 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:49.998982 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:50.000541 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:50.001511 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:50.001679 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:50.005750 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:50.005761 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:50.005779 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:50.005821 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:50.009778 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:50.010493 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:50.010502 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:50.010510 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:50.010721 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:50.011150 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:50.011155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:50.011158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:50.011836 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:50.012117 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:50.012584 (+ 467us) server_negotiation.cc:299] Negotiation successful
0618 22:37:50.013017 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":464,"threads_started":1}
I20260618 22:37:50.016139 7336 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9f3282c784469a8d6b083650afc8e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35537 } }
I20260618 22:37:50.016636 7336 leader_election.cc:302] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce9f3282c784469a8d6b083650afc8e4; no voters:
I20260618 22:37:50.017184 7336 leader_election.cc:288] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:50.017313 7341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:50.017976 7341 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [term 1 LEADER]: Becoming Leader. State: Replica: ce9f3282c784469a8d6b083650afc8e4, State: Running, Role: LEADER
I20260618 22:37:50.018316 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:37:50.018648 7341 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9f3282c784469a8d6b083650afc8e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35537 } }
I20260618 22:37:50.019742 7336 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:50.020527 7341 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce9f3282c784469a8d6b083650afc8e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9f3282c784469a8d6b083650afc8e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35537 } } }
I20260618 22:37:50.020794 7341 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:50.022433 7341 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9f3282c784469a8d6b083650afc8e4. Latest consensus state: current_term: 1 leader_uuid: "ce9f3282c784469a8d6b083650afc8e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9f3282c784469a8d6b083650afc8e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35537 } } }
I20260618 22:37:50.022650 7341 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:50.027127 7346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:50.031850 7346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:50.059068 7346 catalog_manager.cc:1347] Generated new cluster ID: b837b3a203b84f659ea45f5715229fd8
I20260618 22:37:50.059191 7346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:50.059512 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:50.040808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54021 (local address 127.0.0.1:35537)
0618 22:37:50.041000 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:50.041006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:50.041898 (+ 892us) server_negotiation.cc:407] Connection header received
0618 22:37:50.042155 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:50.042162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:50.042251 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:50.042336 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:50.044546 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:50.045747 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:50.047737 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:50.048469 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:50.048568 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:50.052057 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:50.052064 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:50.052070 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:50.052113 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:50.055858 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:50.056756 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:50.056762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:50.056765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:50.056914 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:50.057542 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:50.057548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:50.057552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:50.058032 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:50.058264 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:50.058900 (+ 636us) server_negotiation.cc:299] Negotiation successful
0618 22:37:50.059142 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:37:50.079449 7346 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:50.080842 7346 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:50.094122 7346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce9f3282c784469a8d6b083650afc8e4: Generated new TSK 0
I20260618 22:37:50.094985 7346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:50.101353 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7269
I20260618 22:37:50.102559 7331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:50.196002 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 7269
2026-06-18T22:37:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1894 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:50 dist-test-slave-h7jz krb5kdc[7368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:50 dist-test-slave-h7jz krb5kdc[7368](info): set up 2 sockets
Jun 18 22:37:50 dist-test-slave-h7jz krb5kdc[7368](info): commencing operation
krb5kdc: starting...
W20260618 22:37:51.206070 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:51 dist-test-slave-h7jz krb5kdc[7368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:37:51.438761 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42435
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:51.751788 7381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:51.752059 7381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:51.752146 7381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:51.759377 7381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:51.759485 7381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:51.759534 7381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:51.759594 7381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:51.759636 7381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:51.769316 7381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:51.770560 7381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:51.771793 7381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:51.782859 7387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:51.782918 7389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:51.782943 7386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:51.783881 7381 server_base.cc:1034] running on GCE node
I20260618 22:37:51.784662 7381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:51.786109 7381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:51.787307 7381 hybrid_clock.cc:648] HybridClock initialized: now 1781822271787260 us; error 42 us; skew 500 ppm
I20260618 22:37:51.787634 7381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:51 dist-test-slave-h7jz krb5kdc[7368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822271, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:51.790634 7381 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:37:51.792053 7381 webserver.cc:460] Webserver started at http://127.0.0.1:45279/ using document root <none> and password file <none>
I20260618 22:37:51.792702 7381 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:51.792812 7381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:51.793092 7381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:51.795945 7381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "3a635fa91f9a4d5ea8ad68337f51c08a"
format_stamp: "Formatted at 2026-06-18 22:37:51 on dist-test-slave-h7jz"
server_key: "ef2a040c23212395ee97edb7bfbb88c4"
server_key_iv: "620a91830c61027272c4ae3e142c37d9"
server_key_version: "encryptionkey@0"
I20260618 22:37:51.796657 7381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "3a635fa91f9a4d5ea8ad68337f51c08a"
format_stamp: "Formatted at 2026-06-18 22:37:51 on dist-test-slave-h7jz"
server_key: "ef2a040c23212395ee97edb7bfbb88c4"
server_key_iv: "620a91830c61027272c4ae3e142c37d9"
server_key_version: "encryptionkey@0"
I20260618 22:37:51.801613 7381 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260618 22:37:51.805120 7396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:51.806502 7381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:37:51.806658 7381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "3a635fa91f9a4d5ea8ad68337f51c08a"
format_stamp: "Formatted at 2026-06-18 22:37:51 on dist-test-slave-h7jz"
server_key: "ef2a040c23212395ee97edb7bfbb88c4"
server_key_iv: "620a91830c61027272c4ae3e142c37d9"
server_key_version: "encryptionkey@0"
I20260618 22:37:51.806841 7381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:51.819806 7381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:51.823860 7381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:51.824126 7381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:51.841754 7381 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42435
I20260618 22:37:51.841780 7447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42435 every 8 connection(s)
I20260618 22:37:51.843199 7381 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:51.847009 7448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:51.850315 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 7381
I20260618 22:37:51.850497 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:51.850922 19034 external_mini_cluster.cc:1428] Setting key c5002e26090b09bfc4bdc79d9591a2ee
I20260618 22:37:51.858824 7448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a: Bootstrap starting.
I20260618 22:37:51.861795 7448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:51.862891 7448 log.cc:824] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:51.864133 7451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:51.852821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42367 (local address 127.0.0.1:42435)
0618 22:37:51.853603 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:51.853615 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:51.853656 (+ 41us) server_negotiation.cc:407] Connection header received
0618 22:37:51.855407 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:51.855438 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:51.855880 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:51.856398 (+ 518us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:51.856959 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:51.858325 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:51.859650 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:51.860334 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:51.860507 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:51.861590 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:51.861597 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:51.861611 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:51.862204 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:51.862278 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:51.862308 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:51.862402 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:51.862636 (+ 234us) server_negotiation.cc:299] Negotiation successful
0618 22:37:51.862895 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":339,"threads_started":1}
I20260618 22:37:51.866124 7448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a: No bootstrap required, opened a new log
I20260618 22:37:51.868001 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:37:51.874359 7448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a635fa91f9a4d5ea8ad68337f51c08a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260618 22:37:51.874926 7448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:51.875041 7448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a635fa91f9a4d5ea8ad68337f51c08a, State: Initialized, Role: FOLLOWER
I20260618 22:37:51.875725 7448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a635fa91f9a4d5ea8ad68337f51c08a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260618 22:37:51.876008 7448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:51.876143 7448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:51.876328 7448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:51.878731 7451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:51.872156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47733 (local address 127.0.0.1:42435)
0618 22:37:51.872379 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:51.872385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:51.872740 (+ 355us) server_negotiation.cc:407] Connection header received
0618 22:37:51.872922 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:51.872926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:51.872989 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:51.873074 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:51.873668 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:51.874411 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:51.876323 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:51.876795 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:51.876897 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:51.878016 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:51.878021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:51.878024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:51.878045 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:51.878100 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:51.878116 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:51.878187 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:51.878369 (+ 182us) server_negotiation.cc:299] Negotiation successful
0618 22:37:51.878443 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260618 22:37:51.879302 7448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a635fa91f9a4d5ea8ad68337f51c08a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260618 22:37:51.879854 7448 leader_election.cc:302] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a635fa91f9a4d5ea8ad68337f51c08a; no voters:
I20260618 22:37:51.880470 7448 leader_election.cc:288] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:51.880553 7458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:51.881232 7458 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [term 1 LEADER]: Becoming Leader. State: Replica: 3a635fa91f9a4d5ea8ad68337f51c08a, State: Running, Role: LEADER
I20260618 22:37:51.881772 7458 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a635fa91f9a4d5ea8ad68337f51c08a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260618 22:37:51.882552 7448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:51.884423 7459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a635fa91f9a4d5ea8ad68337f51c08a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a635fa91f9a4d5ea8ad68337f51c08a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } } }
I20260618 22:37:51.884742 7459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:51.886659 7460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a635fa91f9a4d5ea8ad68337f51c08a. Latest consensus state: current_term: 1 leader_uuid: "3a635fa91f9a4d5ea8ad68337f51c08a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a635fa91f9a4d5ea8ad68337f51c08a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } } }
I20260618 22:37:51.887055 7460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:51.888018 7464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:51.893309 7464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:51.902096 7464 catalog_manager.cc:1347] Generated new cluster ID: 6bd26d052fab4bb1a1a4abf11b51223a
I20260618 22:37:51.902245 7464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:51.916877 7464 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:51.918821 7464 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:51.929499 7464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a635fa91f9a4d5ea8ad68337f51c08a: Generated new TSK 0
I20260618 22:37:51.930595 7464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:51.941087 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7381
W20260618 22:37:51.942281 7443 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42367
W20260618 22:37:51.942929 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7381: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:37:51.943042 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 7381
2026-06-18T22:37:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1746 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:52 dist-test-slave-h7jz krb5kdc[7479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:52 dist-test-slave-h7jz krb5kdc[7479](info): set up 2 sockets
Jun 18 22:37:52 dist-test-slave-h7jz krb5kdc[7479](info): commencing operation
krb5kdc: starting...
W20260618 22:37:52.941928 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:53 dist-test-slave-h7jz krb5kdc[7479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:37:53.180847 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37139
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:53.497574 7492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:53.497864 7492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:53.497977 7492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:53.505004 7492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:53.505112 7492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:53.505162 7492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:53.505220 7492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:53.505259 7492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:53.514786 7492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:53.515952 7492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:53.517165 7492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:53.527984 7497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:53.528014 7498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:53.528041 7500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:53.528774 7492 server_base.cc:1034] running on GCE node
I20260618 22:37:53.529479 7492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:53.530823 7492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:53.532027 7492 hybrid_clock.cc:648] HybridClock initialized: now 1781822273531985 us; error 32 us; skew 500 ppm
I20260618 22:37:53.532348 7492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:53 dist-test-slave-h7jz krb5kdc[7479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822273, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:53.538338 7492 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:37:53.539942 7492 webserver.cc:460] Webserver started at http://127.0.0.1:39745/ using document root <none> and password file <none>
I20260618 22:37:53.540573 7492 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:53.540685 7492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:53.540974 7492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:53.543937 7492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "60006b6501804bc89c5bf7d8d4d75c54"
format_stamp: "Formatted at 2026-06-18 22:37:53 on dist-test-slave-h7jz"
server_key: "3497d323eb1b47b24706083d22bf25e5"
server_key_iv: "31717a9ab201dbc452d64316360bde7b"
server_key_version: "encryptionkey@0"
I20260618 22:37:53.544648 7492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "60006b6501804bc89c5bf7d8d4d75c54"
format_stamp: "Formatted at 2026-06-18 22:37:53 on dist-test-slave-h7jz"
server_key: "3497d323eb1b47b24706083d22bf25e5"
server_key_iv: "31717a9ab201dbc452d64316360bde7b"
server_key_version: "encryptionkey@0"
I20260618 22:37:53.549520 7492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260618 22:37:53.552802 7507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:53.554112 7492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:37:53.554256 7492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "60006b6501804bc89c5bf7d8d4d75c54"
format_stamp: "Formatted at 2026-06-18 22:37:53 on dist-test-slave-h7jz"
server_key: "3497d323eb1b47b24706083d22bf25e5"
server_key_iv: "31717a9ab201dbc452d64316360bde7b"
server_key_version: "encryptionkey@0"
I20260618 22:37:53.554440 7492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:53.559691 7492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:53.563803 7492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:53.564069 7492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:53.580675 7492 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37139
I20260618 22:37:53.580720 7558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37139 every 8 connection(s)
I20260618 22:37:53.582018 7492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:53.583361 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 7492
I20260618 22:37:53.583535 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:53.584084 19034 external_mini_cluster.cc:1428] Setting key 1ebdf909c1316d986d2c221708950fcf
I20260618 22:37:53.586791 7559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:53.592898 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:53.585824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59799 (local address 127.0.0.1:37139)
0618 22:37:53.586650 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:53.586665 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:53.586724 (+ 59us) server_negotiation.cc:407] Connection header received
0618 22:37:53.588937 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:53.588966 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:53.589422 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:53.589899 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:53.590904 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:53.590911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:53.590929 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:53.590946 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:53.591035 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:53.591071 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:53.591276 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:53.591421 (+ 145us) server_negotiation.cc:299] Negotiation successful
0618 22:37:53.591615 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":296,"threads_started":1}
I20260618 22:37:53.596494 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:37:53.600680 7559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54: Bootstrap starting.
I20260618 22:37:53.604144 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:53.601658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47907 (local address 127.0.0.1:37139)
0618 22:37:53.601830 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:53.601836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:53.602330 (+ 494us) server_negotiation.cc:407] Connection header received
0618 22:37:53.602504 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:53.602508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:53.602557 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:53.602636 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:53.603534 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:53.603538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:53.603541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:53.603556 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:53.603596 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:53.603611 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:53.603698 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:53.603842 (+ 144us) server_negotiation.cc:299] Negotiation successful
0618 22:37:53.603888 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:37:53.604647 7559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:53.605800 7559 log.cc:824] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:53.608526 7559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54: No bootstrap required, opened a new log
I20260618 22:37:53.615895 7559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60006b6501804bc89c5bf7d8d4d75c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37139 } }
I20260618 22:37:53.616285 7559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:53.616397 7559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60006b6501804bc89c5bf7d8d4d75c54, State: Initialized, Role: FOLLOWER
I20260618 22:37:53.617091 7559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60006b6501804bc89c5bf7d8d4d75c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37139 } }
I20260618 22:37:53.617302 7559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:53.617403 7559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:53.617522 7559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:53.619645 7559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60006b6501804bc89c5bf7d8d4d75c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37139 } }
I20260618 22:37:53.620097 7559 leader_election.cc:302] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60006b6501804bc89c5bf7d8d4d75c54; no voters:
I20260618 22:37:53.620635 7559 leader_election.cc:288] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:53.620785 7569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:53.621402 7569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [term 1 LEADER]: Becoming Leader. State: Replica: 60006b6501804bc89c5bf7d8d4d75c54, State: Running, Role: LEADER
I20260618 22:37:53.621899 7569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60006b6501804bc89c5bf7d8d4d75c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37139 } }
I20260618 22:37:53.622684 7559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:53.624519 7571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60006b6501804bc89c5bf7d8d4d75c54. Latest consensus state: current_term: 1 leader_uuid: "60006b6501804bc89c5bf7d8d4d75c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60006b6501804bc89c5bf7d8d4d75c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37139 } } }
I20260618 22:37:53.624969 7571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:53.625133 7570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60006b6501804bc89c5bf7d8d4d75c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60006b6501804bc89c5bf7d8d4d75c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37139 } } }
I20260618 22:37:53.625465 7570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54 [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:53.627216 7575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:53.632263 7575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:53.647483 7575 catalog_manager.cc:1347] Generated new cluster ID: 2c5806d71361489ba51f52ae6f6febd8
I20260618 22:37:53.647601 7575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:53.709244 7575 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:53.709653 7575 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:53.727603 7575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60006b6501804bc89c5bf7d8d4d75c54: Generated new TSK 0
I20260618 22:37:53.728902 7575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:53.738225 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7492
W20260618 22:37:53.739390 7554 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59799
W20260618 22:37:53.740144 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7492: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:37:53.740296 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 7492
2026-06-18T22:37:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1797 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:53 dist-test-slave-h7jz krb5kdc[7590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:53 dist-test-slave-h7jz krb5kdc[7590](info): set up 2 sockets
Jun 18 22:37:53 dist-test-slave-h7jz krb5kdc[7590](info): commencing operation
krb5kdc: starting...
W20260618 22:37:54.750006 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:54 dist-test-slave-h7jz krb5kdc[7590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:37:54.988441 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35133
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:55.305770 7603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:55.306092 7603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:55.306238 7603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:55.313388 7603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:55.313512 7603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:55.313593 7603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:55.313680 7603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:55.313741 7603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:55.323484 7603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:55.324744 7603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:55.326066 7603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:55.340915 7611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:55.340984 7609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:55.341148 7608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:55.341753 7603 server_base.cc:1034] running on GCE node
I20260618 22:37:55.342525 7603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:55.343924 7603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:55.345124 7603 hybrid_clock.cc:648] HybridClock initialized: now 1781822275345085 us; error 29 us; skew 500 ppm
I20260618 22:37:55.345513 7603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:55 dist-test-slave-h7jz krb5kdc[7590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822275, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:55.348616 7603 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:37:55.350106 7603 webserver.cc:460] Webserver started at http://127.0.0.1:36163/ using document root <none> and password file <none>
I20260618 22:37:55.350780 7603 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:55.350889 7603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:55.351192 7603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:55.354123 7603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "72a439c97f334e13856afaf86eec7d7d"
format_stamp: "Formatted at 2026-06-18 22:37:55 on dist-test-slave-h7jz"
server_key: "696aa7533b51e65645e6e171c0938e54"
server_key_iv: "ed357cdf102ac12bfd1f3d31e2aeb5dd"
server_key_version: "encryptionkey@0"
I20260618 22:37:55.354871 7603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "72a439c97f334e13856afaf86eec7d7d"
format_stamp: "Formatted at 2026-06-18 22:37:55 on dist-test-slave-h7jz"
server_key: "696aa7533b51e65645e6e171c0938e54"
server_key_iv: "ed357cdf102ac12bfd1f3d31e2aeb5dd"
server_key_version: "encryptionkey@0"
I20260618 22:37:55.359768 7603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260618 22:37:55.363169 7618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:55.364439 7603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260618 22:37:55.364585 7603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "72a439c97f334e13856afaf86eec7d7d"
format_stamp: "Formatted at 2026-06-18 22:37:55 on dist-test-slave-h7jz"
server_key: "696aa7533b51e65645e6e171c0938e54"
server_key_iv: "ed357cdf102ac12bfd1f3d31e2aeb5dd"
server_key_version: "encryptionkey@0"
I20260618 22:37:55.364755 7603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:55.383680 7603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:55.387858 7603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:55.388110 7603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:55.404987 7603 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35133
I20260618 22:37:55.405006 7669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35133 every 8 connection(s)
I20260618 22:37:55.406334 7603 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:55.409821 7670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:55.411688 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 7603
I20260618 22:37:55.411881 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:55.412395 19034 external_mini_cluster.cc:1428] Setting key 43408d79117bcc7c6fcccb5beab9a47e
I20260618 22:37:55.421918 7670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d: Bootstrap starting.
I20260618 22:37:55.424860 7670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d: Neither blocks nor log segments found. Creating new log.
Jun 18 22:37:55 dist-test-slave-h7jz krb5kdc[7590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:37:55.425900 7670 log.cc:824] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:55.428599 7670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d: No bootstrap required, opened a new log
I20260618 22:37:55.432902 7673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:55.414225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:46993 (local address 127.0.0.1:35133)
0618 22:37:55.414939 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:55.414951 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:55.414993 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:37:55.416525 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:55.416549 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:55.416937 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:55.417266 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:55.418723 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:55.420017 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:55.421240 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:55.421896 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:55.422091 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:55.426120 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:55.426128 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:55.426141 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:55.426176 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:55.428977 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:55.429710 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:55.429717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:55.429723 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:55.429891 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:55.430386 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:55.430390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:55.430392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:55.430909 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:55.431232 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:55.431665 (+ 433us) server_negotiation.cc:299] Negotiation successful
0618 22:37:55.431953 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":294,"threads_started":1}
I20260618 22:37:55.436836 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:37:55.436637 7670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a439c97f334e13856afaf86eec7d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260618 22:37:55.437001 7670 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:55.437090 7670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a439c97f334e13856afaf86eec7d7d, State: Initialized, Role: FOLLOWER
I20260618 22:37:55.437601 7670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a439c97f334e13856afaf86eec7d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260618 22:37:55.437804 7670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:55.437923 7670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:55.438086 7670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:55.440485 7670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a439c97f334e13856afaf86eec7d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260618 22:37:55.441056 7670 leader_election.cc:302] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72a439c97f334e13856afaf86eec7d7d; no voters:
I20260618 22:37:55.441720 7670 leader_election.cc:288] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:55.441829 7680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:55.443970 7670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:55.448454 7680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [term 1 LEADER]: Becoming Leader. State: Replica: 72a439c97f334e13856afaf86eec7d7d, State: Running, Role: LEADER
I20260618 22:37:55.449112 7680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a439c97f334e13856afaf86eec7d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260618 22:37:55.452252 7682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72a439c97f334e13856afaf86eec7d7d. Latest consensus state: current_term: 1 leader_uuid: "72a439c97f334e13856afaf86eec7d7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a439c97f334e13856afaf86eec7d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } } }
I20260618 22:37:55.452615 7682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:55.453476 7692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:55.453836 7681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72a439c97f334e13856afaf86eec7d7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a439c97f334e13856afaf86eec7d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } } }
I20260618 22:37:55.454169 7681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:55.457029 7673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:55.441681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:34487 (local address 127.0.0.1:35133)
0618 22:37:55.441865 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:55.441871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:55.441889 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:37:55.442204 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:55.442210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:55.442292 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:55.442363 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:55.443900 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:55.444708 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:55.446052 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:55.446493 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:55.446599 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:55.448853 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:55.448858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:55.448861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:37:55.448892 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:55.452503 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:55.453712 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:55.453717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:55.453721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:55.453866 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:37:55.454695 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:37:55.454700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:37:55.454704 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:37:55.455128 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:55.455366 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:55.456453 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0618 22:37:55.456640 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:37:55.469367 7692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:55.477452 7692 catalog_manager.cc:1347] Generated new cluster ID: 7e898c24b5ec4520b5e3149670d1a017
I20260618 22:37:55.477555 7692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:55.488812 7692 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:55.490633 7692 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:55.510703 7692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72a439c97f334e13856afaf86eec7d7d: Generated new TSK 0
I20260618 22:37:55.511795 7692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:55.517252 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7603
I20260618 22:37:55.518474 7665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:37:55.609339 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 7603
W20260618 22:37:55.622836 7588 connection.cc:535] client connection to 127.0.0.1:35133 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35133): Connection reset by peer (error 104)
2026-06-18T22:37:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1868 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:55 dist-test-slave-h7jz krb5kdc[7702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:55 dist-test-slave-h7jz krb5kdc[7702](info): set up 2 sockets
Jun 18 22:37:55 dist-test-slave-h7jz krb5kdc[7702](info): commencing operation
krb5kdc: starting...
W20260618 22:37:56.618149 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:56 dist-test-slave-h7jz krb5kdc[7702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:37:56.851408 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38167
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:57.164337 7715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:57.164615 7715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:57.164755 7715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:57.172041 7715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:57.172153 7715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:57.172233 7715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:57.172313 7715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:57.172361 7715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:57.182061 7715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:57.183319 7715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:57.184579 7715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:57.195766 7720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:57.195809 7723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:57.195828 7721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:57.196707 7715 server_base.cc:1034] running on GCE node
I20260618 22:37:57.197454 7715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:57.198812 7715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:57.199996 7715 hybrid_clock.cc:648] HybridClock initialized: now 1781822277199950 us; error 35 us; skew 500 ppm
I20260618 22:37:57.200309 7715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:57 dist-test-slave-h7jz krb5kdc[7702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822277, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:57.203202 7715 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:37:57.204540 7715 webserver.cc:460] Webserver started at http://127.0.0.1:43979/ using document root <none> and password file <none>
I20260618 22:37:57.205145 7715 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:57.205243 7715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:57.205487 7715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:57.208287 7715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "5f0094656ae44895a47f5868266baefb"
format_stamp: "Formatted at 2026-06-18 22:37:57 on dist-test-slave-h7jz"
server_key: "fe141177c1e580b41a4ba258c9dd9451"
server_key_iv: "8ee05263509a56cae7cfd2c29cc46132"
server_key_version: "encryptionkey@0"
I20260618 22:37:57.208978 7715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "5f0094656ae44895a47f5868266baefb"
format_stamp: "Formatted at 2026-06-18 22:37:57 on dist-test-slave-h7jz"
server_key: "fe141177c1e580b41a4ba258c9dd9451"
server_key_iv: "8ee05263509a56cae7cfd2c29cc46132"
server_key_version: "encryptionkey@0"
I20260618 22:37:57.213785 7715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260618 22:37:57.217101 7730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:57.218444 7715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:37:57.218587 7715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "5f0094656ae44895a47f5868266baefb"
format_stamp: "Formatted at 2026-06-18 22:37:57 on dist-test-slave-h7jz"
server_key: "fe141177c1e580b41a4ba258c9dd9451"
server_key_iv: "8ee05263509a56cae7cfd2c29cc46132"
server_key_version: "encryptionkey@0"
I20260618 22:37:57.218762 7715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:57.243971 7715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:57.248416 7715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:57.248692 7715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:57.264569 7715 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38167
I20260618 22:37:57.264600 7781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38167 every 8 connection(s)
I20260618 22:37:57.266024 7715 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:57.269685 7782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:57.274123 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 7715
I20260618 22:37:57.274312 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:57.274729 19034 external_mini_cluster.cc:1428] Setting key d43e3b5debcfaa9e30618872e3f7be7b
I20260618 22:37:57.280155 7782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb: Bootstrap starting.
I20260618 22:37:57.283595 7782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:57.284703 7782 log.cc:824] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:57.287585 7782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb: No bootstrap required, opened a new log
I20260618 22:37:57.288030 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:57.276445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:54427 (local address 127.0.0.1:38167)
0618 22:37:57.277182 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:57.277194 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:57.277238 (+ 44us) server_negotiation.cc:407] Connection header received
0618 22:37:57.279000 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:57.279024 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:57.279432 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:57.279761 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:57.280352 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:57.281649 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:57.283565 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:57.284320 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:57.284460 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:57.285769 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:57.285781 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:57.285799 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:57.286392 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:57.286461 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:57.286486 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:57.286603 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:57.286862 (+ 259us) server_negotiation.cc:299] Negotiation successful
0618 22:37:57.287027 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":321,"threads_started":1}
I20260618 22:37:57.291834 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:37:57.295490 7782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0094656ae44895a47f5868266baefb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } }
I20260618 22:37:57.295883 7782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:57.295987 7782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f0094656ae44895a47f5868266baefb, State: Initialized, Role: FOLLOWER
I20260618 22:37:57.296674 7782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0094656ae44895a47f5868266baefb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } }
I20260618 22:37:57.296936 7782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:57.297079 7782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:57.297240 7782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:57.300204 7782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0094656ae44895a47f5868266baefb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } }
I20260618 22:37:57.300678 7782 leader_election.cc:302] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f0094656ae44895a47f5868266baefb; no voters:
I20260618 22:37:57.301295 7782 leader_election.cc:288] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:57.301403 7792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:57.302740 7792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [term 1 LEADER]: Becoming Leader. State: Replica: 5f0094656ae44895a47f5868266baefb, State: Running, Role: LEADER
I20260618 22:37:57.303079 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:57.295926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:43409 (local address 127.0.0.1:38167)
0618 22:37:57.296112 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:57.296117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:57.296477 (+ 360us) server_negotiation.cc:407] Connection header received
0618 22:37:57.296645 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:57.296648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:57.296710 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:57.296782 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:57.297412 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:57.298303 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:57.300100 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:37:57.300610 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:37:57.300735 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:37:57.302051 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:57.302057 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:57.302061 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:57.302082 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:57.302145 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:57.302168 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:57.302263 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:57.302682 (+ 419us) server_negotiation.cc:299] Negotiation successful
0618 22:37:57.302732 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:37:57.303277 7792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0094656ae44895a47f5868266baefb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } }
I20260618 22:37:57.303491 7782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:57.307415 7793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f0094656ae44895a47f5868266baefb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0094656ae44895a47f5868266baefb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } } }
I20260618 22:37:57.307826 7793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:57.308225 7794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f0094656ae44895a47f5868266baefb. Latest consensus state: current_term: 1 leader_uuid: "5f0094656ae44895a47f5868266baefb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0094656ae44895a47f5868266baefb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } } }
I20260618 22:37:57.308482 7794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:57.309894 7805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:57.313913 7805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:57.322299 7805 catalog_manager.cc:1347] Generated new cluster ID: 1d53889d069842d3b25b27a647720df9
I20260618 22:37:57.322405 7805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:57.335675 7805 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:57.336951 7805 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:57.346038 7805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f0094656ae44895a47f5868266baefb: Generated new TSK 0
I20260618 22:37:57.346839 7805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:57.355650 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7715
W20260618 22:37:57.356813 7777 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:54427
W20260618 22:37:57.357477 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7715: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:37:57.357627 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 7715
W20260618 22:37:57.375874 7700 connection.cc:535] client connection to 127.0.0.1:38167 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38167): Connection reset by peer (error 104)
2026-06-18T22:37:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1753 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:57 dist-test-slave-h7jz krb5kdc[7813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:57 dist-test-slave-h7jz krb5kdc[7813](info): set up 2 sockets
Jun 18 22:37:57 dist-test-slave-h7jz krb5kdc[7813](info): commencing operation
krb5kdc: starting...
W20260618 22:37:58.369969 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:37:58 dist-test-slave-h7jz krb5kdc[7813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:37:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:37:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:37:58.602013 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40775
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:37:58.921599 7826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:37:58.921905 7826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:37:58.922052 7826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:37:58.929591 7826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:37:58.929757 7826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:37:58.929852 7826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:37:58.929968 7826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:37:58.930037 7826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:37:58.940225 7826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:37:58.941546 7826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:37:58.942930 7826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:37:58.954727 7831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:58.954766 7834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:37:58.954797 7832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:37:58.955650 7826 server_base.cc:1034] running on GCE node
I20260618 22:37:58.956423 7826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:37:58.957828 7826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:37:58.959043 7826 hybrid_clock.cc:648] HybridClock initialized: now 1781822278959005 us; error 59 us; skew 500 ppm
I20260618 22:37:58.959398 7826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:37:58 dist-test-slave-h7jz krb5kdc[7813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822278, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:37:58.967753 7826 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:37:58.969255 7826 webserver.cc:460] Webserver started at http://127.0.0.1:39683/ using document root <none> and password file <none>
I20260618 22:37:58.969962 7826 fs_manager.cc:359] Metadata directory not provided
I20260618 22:37:58.970089 7826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:37:58.970391 7826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:37:58.973347 7826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "8196a0427a2e44ab99d6c5b2ce20f66c"
format_stamp: "Formatted at 2026-06-18 22:37:58 on dist-test-slave-h7jz"
server_key: "062f318cbebb3956d3017bab0d633629"
server_key_iv: "0a2a9fcfa116eb1fec4f988c4a1b314e"
server_key_version: "encryptionkey@0"
I20260618 22:37:58.974126 7826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "8196a0427a2e44ab99d6c5b2ce20f66c"
format_stamp: "Formatted at 2026-06-18 22:37:58 on dist-test-slave-h7jz"
server_key: "062f318cbebb3956d3017bab0d633629"
server_key_iv: "0a2a9fcfa116eb1fec4f988c4a1b314e"
server_key_version: "encryptionkey@0"
I20260618 22:37:58.978993 7826 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260618 22:37:58.982277 7841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:37:58.983525 7826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:37:58.983666 7826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "8196a0427a2e44ab99d6c5b2ce20f66c"
format_stamp: "Formatted at 2026-06-18 22:37:58 on dist-test-slave-h7jz"
server_key: "062f318cbebb3956d3017bab0d633629"
server_key_iv: "0a2a9fcfa116eb1fec4f988c4a1b314e"
server_key_version: "encryptionkey@0"
I20260618 22:37:58.983830 7826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:37:58.989355 7826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:37:58.993340 7826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:37:58.993614 7826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:37:59.010438 7826 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40775
I20260618 22:37:59.010483 7892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40775 every 8 connection(s)
I20260618 22:37:59.011763 7826 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:37:59.014458 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 7826
I20260618 22:37:59.014640 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:37:59.015174 19034 external_mini_cluster.cc:1428] Setting key 2c051ba69491137cf92b518127491c03
I20260618 22:37:59.015799 7893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:37:59.022657 7896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:59.016952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:49505 (local address 127.0.0.1:40775)
0618 22:37:59.017792 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:59.017805 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:59.017844 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:37:59.019345 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:59.019366 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:59.019702 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:59.020074 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:59.021087 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:59.021093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:59.021110 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:59.021125 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:59.021198 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:59.021220 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:59.021428 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:59.021557 (+ 129us) server_negotiation.cc:299] Negotiation successful
0618 22:37:59.021736 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":435,"threads_started":1}
I20260618 22:37:59.026021 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:37:59.028695 7893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c: Bootstrap starting.
I20260618 22:37:59.032402 7896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:37:59.030130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:45795 (local address 127.0.0.1:40775)
0618 22:37:59.030298 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0618 22:37:59.030303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:37:59.030599 (+ 296us) server_negotiation.cc:407] Connection header received
0618 22:37:59.030818 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:37:59.030822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:37:59.030864 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:37:59.030932 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:37:59.031859 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:37:59.031863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:37:59.031866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:37:59.031880 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:37:59.031915 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:37:59.031930 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:37:59.032007 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0618 22:37:59.032134 (+ 127us) server_negotiation.cc:299] Negotiation successful
0618 22:37:59.032176 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260618 22:37:59.032538 7893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c: Neither blocks nor log segments found. Creating new log.
I20260618 22:37:59.033802 7893 log.cc:824] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:37:59.036612 7893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c: No bootstrap required, opened a new log
I20260618 22:37:59.043787 7893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8196a0427a2e44ab99d6c5b2ce20f66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260618 22:37:59.044133 7893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:37:59.044211 7893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8196a0427a2e44ab99d6c5b2ce20f66c, State: Initialized, Role: FOLLOWER
I20260618 22:37:59.044687 7893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8196a0427a2e44ab99d6c5b2ce20f66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260618 22:37:59.044869 7893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:37:59.044981 7893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:37:59.045097 7893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:37:59.047271 7893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8196a0427a2e44ab99d6c5b2ce20f66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260618 22:37:59.047767 7893 leader_election.cc:302] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8196a0427a2e44ab99d6c5b2ce20f66c; no voters:
I20260618 22:37:59.048348 7893 leader_election.cc:288] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:37:59.048449 7903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:37:59.049351 7903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [term 1 LEADER]: Becoming Leader. State: Replica: 8196a0427a2e44ab99d6c5b2ce20f66c, State: Running, Role: LEADER
I20260618 22:37:59.049813 7903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8196a0427a2e44ab99d6c5b2ce20f66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260618 22:37:59.050158 7893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:37:59.052557 7904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8196a0427a2e44ab99d6c5b2ce20f66c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8196a0427a2e44ab99d6c5b2ce20f66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } } }
I20260618 22:37:59.052525 7905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8196a0427a2e44ab99d6c5b2ce20f66c. Latest consensus state: current_term: 1 leader_uuid: "8196a0427a2e44ab99d6c5b2ce20f66c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8196a0427a2e44ab99d6c5b2ce20f66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } } }
I20260618 22:37:59.053040 7904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:59.053040 7905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c [sys.catalog]: This master's current role is: LEADER
I20260618 22:37:59.058990 7911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:37:59.063884 7911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:37:59.072443 7911 catalog_manager.cc:1347] Generated new cluster ID: 4ee61b4a53e941ae910e4f909d1277e7
I20260618 22:37:59.072530 7911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:37:59.094631 7911 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:37:59.095044 7911 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:37:59.104722 7911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8196a0427a2e44ab99d6c5b2ce20f66c: Generated new TSK 0
I20260618 22:37:59.105718 7911 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:37:59.122038 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7826
W20260618 22:37:59.123133 7888 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:49505
W20260618 22:37:59.123766 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7826: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:37:59.123893 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 7826
2026-06-18T22:37:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1760 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:37:59 dist-test-slave-h7jz krb5kdc[7924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:37:59 dist-test-slave-h7jz krb5kdc[7924](info): set up 2 sockets
Jun 18 22:37:59 dist-test-slave-h7jz krb5kdc[7924](info): commencing operation
krb5kdc: starting...
W20260618 22:38:00.134183 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:38:00 dist-test-slave-h7jz krb5kdc[7924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:38:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:38:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:38:00.358341 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38377
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:38:00.669782 7937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:00.670084 7937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:00.670229 7937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:00.677403 7937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:00.677518 7937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:00.677595 7937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:00.677675 7937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:00.677734 7937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:00.687209 7937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38377
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:00.688457 7937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:00.689699 7937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:00.700598 7942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:00.700531 7945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:00.700559 7943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:00.701602 7937 server_base.cc:1034] running on GCE node
I20260618 22:38:00.702319 7937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:00.703593 7937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:00.704787 7937 hybrid_clock.cc:648] HybridClock initialized: now 1781822280704729 us; error 48 us; skew 500 ppm
I20260618 22:38:00.705107 7937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:00 dist-test-slave-h7jz krb5kdc[7924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822280, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:00.707933 7937 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:38:00.709282 7937 webserver.cc:460] Webserver started at http://127.0.0.1:45311/ using document root <none> and password file <none>
I20260618 22:38:00.709904 7937 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:00.710062 7937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:00.710331 7937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:00.713101 7937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "cfc1f1a4aece44ca8de7e863a8c1339f"
format_stamp: "Formatted at 2026-06-18 22:38:00 on dist-test-slave-h7jz"
server_key: "6140c366561a2eceb0b8a67eb4341e06"
server_key_iv: "36809371b877e202d8003421ecafded6"
server_key_version: "encryptionkey@0"
I20260618 22:38:00.713795 7937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "cfc1f1a4aece44ca8de7e863a8c1339f"
format_stamp: "Formatted at 2026-06-18 22:38:00 on dist-test-slave-h7jz"
server_key: "6140c366561a2eceb0b8a67eb4341e06"
server_key_iv: "36809371b877e202d8003421ecafded6"
server_key_version: "encryptionkey@0"
I20260618 22:38:00.718583 7937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:38:00.722007 7952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:00.723618 7937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:38:00.723806 7937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "cfc1f1a4aece44ca8de7e863a8c1339f"
format_stamp: "Formatted at 2026-06-18 22:38:00 on dist-test-slave-h7jz"
server_key: "6140c366561a2eceb0b8a67eb4341e06"
server_key_iv: "36809371b877e202d8003421ecafded6"
server_key_version: "encryptionkey@0"
I20260618 22:38:00.723973 7937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:00.748857 7937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:00.752723 7937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:00.752971 7937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:00.769919 7937 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38377
I20260618 22:38:00.769989 8003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38377 every 8 connection(s)
I20260618 22:38:00.771348 7937 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:38:00.775107 8004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:00.778916 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 7937
I20260618 22:38:00.779094 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:38:00.779626 19034 external_mini_cluster.cc:1428] Setting key 4b6ae94c7c3004e49a928c549e1e342c
I20260618 22:38:00.787905 8004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f: Bootstrap starting.
I20260618 22:38:00.791604 8004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:00.792869 8004 log.cc:824] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:38:00 dist-test-slave-h7jz krb5kdc[7924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:38:00.795837 8004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f: No bootstrap required, opened a new log
I20260618 22:38:00.802405 8007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:00.781358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:59141 (local address 127.0.0.1:38377)
0618 22:38:00.782056 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:00.782068 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:00.782108 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:38:00.783807 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:00.783837 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:00.784254 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:00.784577 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:00.786516 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:00.787815 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:00.789498 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:00.790238 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:00.790385 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:00.795158 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:00.795166 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:00.795180 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:00.795219 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:00.798200 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:00.798993 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:00.799000 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:00.799006 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:00.799182 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:00.799634 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:00.799637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:00.799640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:00.800177 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:00.800515 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:00.800993 (+ 478us) server_negotiation.cc:299] Negotiation successful
0618 22:38:00.801329 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":283,"threads_started":1}
I20260618 22:38:00.803642 8004 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1f1a4aece44ca8de7e863a8c1339f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38377 } }
I20260618 22:38:00.804004 8004 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:38:00.804093 8004 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfc1f1a4aece44ca8de7e863a8c1339f, State: Initialized, Role: FOLLOWER
I20260618 22:38:00.804642 8004 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1f1a4aece44ca8de7e863a8c1339f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38377 } }
I20260618 22:38:00.804925 8004 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:38:00.805047 8004 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:38:00.805199 8004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:00.806162 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:38:00.807518 8004 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1f1a4aece44ca8de7e863a8c1339f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38377 } }
I20260618 22:38:00.807999 8004 leader_election.cc:302] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfc1f1a4aece44ca8de7e863a8c1339f; no voters:
I20260618 22:38:00.808586 8004 leader_election.cc:288] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:38:00.808739 8009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:38:00.810518 8004 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:00.814419 8009 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [term 1 LEADER]: Becoming Leader. State: Replica: cfc1f1a4aece44ca8de7e863a8c1339f, State: Running, Role: LEADER
I20260618 22:38:00.815092 8009 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1f1a4aece44ca8de7e863a8c1339f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38377 } }
I20260618 22:38:00.818835 8012 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfc1f1a4aece44ca8de7e863a8c1339f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1f1a4aece44ca8de7e863a8c1339f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38377 } } }
I20260618 22:38:00.819226 8012 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:00.819674 8013 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfc1f1a4aece44ca8de7e863a8c1339f. Latest consensus state: current_term: 1 leader_uuid: "cfc1f1a4aece44ca8de7e863a8c1339f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1f1a4aece44ca8de7e863a8c1339f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38377 } } }
I20260618 22:38:00.819929 8013 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:00.823369 8028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:38:00.828832 8028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:38:00.837618 8007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:00.817192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:55939 (local address 127.0.0.1:38377)
0618 22:38:00.817388 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:00.817395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:00.822590 (+ 5195us) server_negotiation.cc:407] Connection header received
0618 22:38:00.823156 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:00.823160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:00.823223 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:00.823381 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:00.824788 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:00.825833 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:00.827171 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:00.827744 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:00.827884 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:00.830327 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:00.830332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:00.830335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:00.830367 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:00.833449 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:00.834476 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:00.834481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:00.834484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:00.834626 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:00.835659 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:00.835662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:00.835665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:00.836038 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:00.836328 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:00.837055 (+ 727us) server_negotiation.cc:299] Negotiation successful
0618 22:38:00.837242 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:38:00.838634 8028 catalog_manager.cc:1347] Generated new cluster ID: d320eb4cbec4464c865d3157cb0da841
I20260618 22:38:00.838790 8028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:38:00.845882 8028 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:38:00.847714 8028 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:38:00.859124 8028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfc1f1a4aece44ca8de7e863a8c1339f: Generated new TSK 0
I20260618 22:38:00.860186 8028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:38:00.865341 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 7937
I20260618 22:38:00.866770 7999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:38:00.954515 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 7937
W20260618 22:38:00.967798 7922 connection.cc:535] client connection to 127.0.0.1:38377 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38377): Connection reset by peer (error 104)
2026-06-18T22:38:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1830 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:38:01 dist-test-slave-h7jz krb5kdc[8036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:38:01 dist-test-slave-h7jz krb5kdc[8036](info): set up 2 sockets
Jun 18 22:38:01 dist-test-slave-h7jz krb5kdc[8036](info): commencing operation
krb5kdc: starting...
W20260618 22:38:01.958120 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.008s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:38:02 dist-test-slave-h7jz krb5kdc[8036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:38:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:38:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:38:02.187377 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34625
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:38:02.502532 8049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:02.502831 8049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:02.502977 8049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:02.510185 8049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:02.510298 8049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:02.510377 8049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:02.510457 8049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:02.510516 8049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:02.520184 8049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34625
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:02.521441 8049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:02.522758 8049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:02.534186 8054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:02.534157 8055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:02.534224 8057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:02.534875 8049 server_base.cc:1034] running on GCE node
I20260618 22:38:02.535588 8049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:02.536971 8049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:02.538161 8049 hybrid_clock.cc:648] HybridClock initialized: now 1781822282538115 us; error 36 us; skew 500 ppm
I20260618 22:38:02.538492 8049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:02 dist-test-slave-h7jz krb5kdc[8036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822282, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:02.541401 8049 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:38:02.542824 8049 webserver.cc:460] Webserver started at http://127.0.0.1:35267/ using document root <none> and password file <none>
I20260618 22:38:02.543486 8049 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:02.543601 8049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:02.543885 8049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:02.546949 8049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "f46d4b863a41471490007d7f028f5be1"
format_stamp: "Formatted at 2026-06-18 22:38:02 on dist-test-slave-h7jz"
server_key: "16fd8b88ac48390cfc2be997ce7719dd"
server_key_iv: "718d77c01d450ebcd7c5eaed3970d3ec"
server_key_version: "encryptionkey@0"
I20260618 22:38:02.547681 8049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "f46d4b863a41471490007d7f028f5be1"
format_stamp: "Formatted at 2026-06-18 22:38:02 on dist-test-slave-h7jz"
server_key: "16fd8b88ac48390cfc2be997ce7719dd"
server_key_iv: "718d77c01d450ebcd7c5eaed3970d3ec"
server_key_version: "encryptionkey@0"
I20260618 22:38:02.552625 8049 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:38:02.555946 8064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:02.557210 8049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:38:02.557366 8049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "f46d4b863a41471490007d7f028f5be1"
format_stamp: "Formatted at 2026-06-18 22:38:02 on dist-test-slave-h7jz"
server_key: "16fd8b88ac48390cfc2be997ce7719dd"
server_key_iv: "718d77c01d450ebcd7c5eaed3970d3ec"
server_key_version: "encryptionkey@0"
I20260618 22:38:02.557534 8049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:02.569905 8049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:02.573869 8049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:02.574123 8049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:02.591024 8049 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34625
I20260618 22:38:02.591045 8115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34625 every 8 connection(s)
I20260618 22:38:02.592370 8049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:38:02.596010 8116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:02.598910 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 8049
I20260618 22:38:02.599073 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:38:02.599515 19034 external_mini_cluster.cc:1428] Setting key 3cd7a1a286621326d601c3bde45d33f7
I20260618 22:38:02.607534 8116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1: Bootstrap starting.
I20260618 22:38:02.610983 8116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:02.612089 8116 log.cc:824] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:02.612524 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:02.601311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:49761 (local address 127.0.0.1:34625)
0618 22:38:02.602036 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:02.602048 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:02.602088 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:38:02.603687 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:02.603709 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:02.604121 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:02.604455 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:02.605009 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:02.606416 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:02.608149 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:02.608844 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:02.608990 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:02.610269 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:02.610277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:02.610290 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:02.610790 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:02.610898 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:02.610922 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:02.611046 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:02.611304 (+ 258us) server_negotiation.cc:299] Negotiation successful
0618 22:38:02.611494 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":293,"threads_started":1}
I20260618 22:38:02.614825 8116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1: No bootstrap required, opened a new log
I20260618 22:38:02.616230 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:38:02.622347 8116 raft_consensus.cc:348] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46d4b863a41471490007d7f028f5be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } }
I20260618 22:38:02.622699 8116 raft_consensus.cc:374] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:38:02.622783 8116 raft_consensus.cc:729] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46d4b863a41471490007d7f028f5be1, State: Initialized, Role: FOLLOWER
I20260618 22:38:02.623345 8116 consensus_queue.cc:260] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46d4b863a41471490007d7f028f5be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } }
I20260618 22:38:02.623564 8116 raft_consensus.cc:388] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:38:02.623690 8116 raft_consensus.cc:482] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:38:02.623852 8116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:02.626056 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:02.620274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:40129 (local address 127.0.0.1:34625)
0618 22:38:02.620450 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:02.620456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:02.620820 (+ 364us) server_negotiation.cc:407] Connection header received
0618 22:38:02.620981 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:02.620984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:02.621047 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:02.621136 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:02.621626 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:02.622404 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:02.623689 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:02.624175 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:02.624277 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:02.625330 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:02.625335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:02.625338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:02.625360 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:02.625416 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:02.625431 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:02.625515 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:02.625703 (+ 188us) server_negotiation.cc:299] Negotiation successful
0618 22:38:02.625754 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:38:02.626118 8116 raft_consensus.cc:504] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46d4b863a41471490007d7f028f5be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } }
I20260618 22:38:02.626705 8116 leader_election.cc:302] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f46d4b863a41471490007d7f028f5be1; no voters:
I20260618 22:38:02.627337 8116 leader_election.cc:288] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:38:02.627466 8126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:38:02.628067 8126 raft_consensus.cc:686] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [term 1 LEADER]: Becoming Leader. State: Replica: f46d4b863a41471490007d7f028f5be1, State: Running, Role: LEADER
I20260618 22:38:02.628558 8126 consensus_queue.cc:237] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46d4b863a41471490007d7f028f5be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } }
I20260618 22:38:02.629331 8116 sys_catalog.cc:564] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:02.631201 8128 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f46d4b863a41471490007d7f028f5be1. Latest consensus state: current_term: 1 leader_uuid: "f46d4b863a41471490007d7f028f5be1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46d4b863a41471490007d7f028f5be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } } }
I20260618 22:38:02.631654 8128 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:02.631582 8127 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f46d4b863a41471490007d7f028f5be1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46d4b863a41471490007d7f028f5be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34625 } } }
I20260618 22:38:02.631857 8127 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:02.633471 8132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:38:02.638954 8132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:38:02.648594 8132 catalog_manager.cc:1347] Generated new cluster ID: 9dcef9250d1d40e1a2486152e6b02f31
I20260618 22:38:02.648952 8132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:38:02.660728 8132 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:38:02.662315 8132 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:38:02.668620 8132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f46d4b863a41471490007d7f028f5be1: Generated new TSK 0
I20260618 22:38:02.669373 8132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:38:02.675997 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8049
W20260618 22:38:02.677398 8111 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:49761
W20260618 22:38:02.678081 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8049: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:38:02.678220 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 8049
W20260618 22:38:02.691541 8034 connection.cc:535] client connection to 127.0.0.1:34625 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34625): Connection reset by peer (error 104)
2026-06-18T22:38:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1723 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:38:02 dist-test-slave-h7jz krb5kdc[8147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:38:02 dist-test-slave-h7jz krb5kdc[8147](info): set up 2 sockets
Jun 18 22:38:02 dist-test-slave-h7jz krb5kdc[8147](info): commencing operation
krb5kdc: starting...
W20260618 22:38:03.694093 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:38:03 dist-test-slave-h7jz krb5kdc[8147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:38:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:38:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:38:03.932680 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42153
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:38:04.250201 8160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:04.250489 8160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:04.250627 8160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:04.257753 8160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:04.257869 8160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:04.257956 8160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:04.258041 8160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:04.258096 8160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:04.267860 8160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42153
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:04.269136 8160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:04.270745 8160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:04.282570 8165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:04.282534 8168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:04.282496 8166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:04.283555 8160 server_base.cc:1034] running on GCE node
I20260618 22:38:04.284358 8160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:04.285826 8160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:04.287050 8160 hybrid_clock.cc:648] HybridClock initialized: now 1781822284286987 us; error 56 us; skew 500 ppm
I20260618 22:38:04.287408 8160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:04 dist-test-slave-h7jz krb5kdc[8147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822284, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:04.293367 8160 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:38:04.294974 8160 webserver.cc:460] Webserver started at http://127.0.0.1:43285/ using document root <none> and password file <none>
I20260618 22:38:04.295674 8160 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:04.295802 8160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:04.296096 8160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:04.299257 8160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "89650c17785748c8b01868b039d641f6"
format_stamp: "Formatted at 2026-06-18 22:38:04 on dist-test-slave-h7jz"
server_key: "6d54d35087f2afe67fca07e0398ccb7b"
server_key_iv: "9c10cf0c98e0f2de623a155271594c8a"
server_key_version: "encryptionkey@0"
I20260618 22:38:04.300163 8160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "89650c17785748c8b01868b039d641f6"
format_stamp: "Formatted at 2026-06-18 22:38:04 on dist-test-slave-h7jz"
server_key: "6d54d35087f2afe67fca07e0398ccb7b"
server_key_iv: "9c10cf0c98e0f2de623a155271594c8a"
server_key_version: "encryptionkey@0"
I20260618 22:38:04.305598 8160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:38:04.309252 8175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:04.310647 8160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:38:04.310819 8160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "89650c17785748c8b01868b039d641f6"
format_stamp: "Formatted at 2026-06-18 22:38:04 on dist-test-slave-h7jz"
server_key: "6d54d35087f2afe67fca07e0398ccb7b"
server_key_iv: "9c10cf0c98e0f2de623a155271594c8a"
server_key_version: "encryptionkey@0"
I20260618 22:38:04.311008 8160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:04.316582 8160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:04.320701 8160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:04.320967 8160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:04.337975 8160 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42153
I20260618 22:38:04.338019 8226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42153 every 8 connection(s)
I20260618 22:38:04.339329 8160 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:38:04.342911 8227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:04.345506 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 8160
I20260618 22:38:04.345665 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:38:04.346128 19034 external_mini_cluster.cc:1428] Setting key 477ef97aadd885cc55e02dca13a6e151
I20260618 22:38:04.353552 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:04.347804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:43007 (local address 127.0.0.1:42153)
0618 22:38:04.348551 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:04.348563 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:04.348602 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:38:04.350290 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:04.350318 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:04.350678 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:04.351021 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:04.352040 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:04.352046 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:04.352061 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:04.352086 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:04.352139 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:04.352160 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:04.352322 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:04.352450 (+ 128us) server_negotiation.cc:299] Negotiation successful
0618 22:38:04.352628 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":319,"threads_started":1}
I20260618 22:38:04.354307 8227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6: Bootstrap starting.
I20260618 22:38:04.357141 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:38:04.357558 8227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:04.358553 8227 log.cc:824] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:04.361604 8227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6: No bootstrap required, opened a new log
I20260618 22:38:04.363903 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:04.361575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:50027 (local address 127.0.0.1:42153)
0618 22:38:04.361739 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:04.361744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:04.362129 (+ 385us) server_negotiation.cc:407] Connection header received
0618 22:38:04.362301 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:04.362305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:04.362351 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:04.362441 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:04.363289 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:04.363294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:04.363302 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:04.363318 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:04.363358 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:04.363380 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:04.363480 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:04.363578 (+ 98us) server_negotiation.cc:299] Negotiation successful
0618 22:38:04.363646 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:38:04.369210 8227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89650c17785748c8b01868b039d641f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42153 } }
I20260618 22:38:04.369626 8227 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:38:04.369724 8227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89650c17785748c8b01868b039d641f6, State: Initialized, Role: FOLLOWER
I20260618 22:38:04.370316 8227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89650c17785748c8b01868b039d641f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42153 } }
I20260618 22:38:04.370539 8227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:38:04.370661 8227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:38:04.370805 8227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:04.373019 8227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89650c17785748c8b01868b039d641f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42153 } }
I20260618 22:38:04.373466 8227 leader_election.cc:302] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89650c17785748c8b01868b039d641f6; no voters:
I20260618 22:38:04.374104 8227 leader_election.cc:288] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:38:04.374219 8237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:38:04.374845 8237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [term 1 LEADER]: Becoming Leader. State: Replica: 89650c17785748c8b01868b039d641f6, State: Running, Role: LEADER
I20260618 22:38:04.375433 8237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89650c17785748c8b01868b039d641f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42153 } }
I20260618 22:38:04.376073 8227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:04.378208 8239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89650c17785748c8b01868b039d641f6. Latest consensus state: current_term: 1 leader_uuid: "89650c17785748c8b01868b039d641f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89650c17785748c8b01868b039d641f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42153 } } }
I20260618 22:38:04.378386 8238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89650c17785748c8b01868b039d641f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89650c17785748c8b01868b039d641f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42153 } } }
I20260618 22:38:04.378613 8239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:04.378677 8238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:04.380280 8245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:38:04.385141 8245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:38:04.393854 8245 catalog_manager.cc:1347] Generated new cluster ID: e43d9287b83e42fa91e7e9d12386b190
I20260618 22:38:04.393986 8245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:38:04.410669 8245 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:38:04.411154 8245 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:38:04.418087 8245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89650c17785748c8b01868b039d641f6: Generated new TSK 0
I20260618 22:38:04.419083 8245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:38:04.432325 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8160
W20260618 22:38:04.434202 8222 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.143:43007
W20260618 22:38:04.434900 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8160: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:38:04.435052 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 8160
2026-06-18T22:38:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1756 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:38:04 dist-test-slave-h7jz krb5kdc[8258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:38:04 dist-test-slave-h7jz krb5kdc[8258](info): set up 2 sockets
Jun 18 22:38:04 dist-test-slave-h7jz krb5kdc[8258](info): commencing operation
krb5kdc: starting...
W20260618 22:38:05.450018 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.005s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:38:05 dist-test-slave-h7jz krb5kdc[8258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:38:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:38:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:38:05.677750 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:42403
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:43001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:42403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:38:05.992491 8271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:05.992787 8271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:05.992923 8271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:05.999974 8271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:06.000094 8271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:06.000174 8271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:06.000257 8271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:06.000314 8271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:06.010015 8271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:43001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:42403
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:42403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:06.011276 8271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:06.012586 8271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:06.023550 8276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:06.023589 8279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:06.023612 8277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:06.024282 8271 server_base.cc:1034] running on GCE node
I20260618 22:38:06.025007 8271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:06.026366 8271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:06.027562 8271 hybrid_clock.cc:648] HybridClock initialized: now 1781822286027519 us; error 34 us; skew 500 ppm
I20260618 22:38:06.027874 8271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:06 dist-test-slave-h7jz krb5kdc[8258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822286, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:06.030805 8271 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:38:06.032176 8271 webserver.cc:460] Webserver started at http://127.18.150.190:34561/ using document root <none> and password file <none>
I20260618 22:38:06.032806 8271 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:06.032915 8271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:06.033188 8271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:06.036016 8271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "923811dee766415b82d5592fb2a9fc81"
format_stamp: "Formatted at 2026-06-18 22:38:06 on dist-test-slave-h7jz"
server_key: "b9f875fd8b08fc0bd9386aed914e9011"
server_key_iv: "6d7ce220114dd5b497b990336dd752ed"
server_key_version: "encryptionkey@0"
I20260618 22:38:06.036720 8271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "923811dee766415b82d5592fb2a9fc81"
format_stamp: "Formatted at 2026-06-18 22:38:06 on dist-test-slave-h7jz"
server_key: "b9f875fd8b08fc0bd9386aed914e9011"
server_key_iv: "6d7ce220114dd5b497b990336dd752ed"
server_key_version: "encryptionkey@0"
I20260618 22:38:06.041492 8271 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.007s
I20260618 22:38:06.044744 8286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:06.046065 8271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:38:06.046216 8271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "923811dee766415b82d5592fb2a9fc81"
format_stamp: "Formatted at 2026-06-18 22:38:06 on dist-test-slave-h7jz"
server_key: "b9f875fd8b08fc0bd9386aed914e9011"
server_key_iv: "6d7ce220114dd5b497b990336dd752ed"
server_key_version: "encryptionkey@0"
I20260618 22:38:06.046375 8271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:06.073238 8271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:06.077426 8271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:06.077680 8271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:06.094604 8271 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:42403
I20260618 22:38:06.094610 8337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:42403 every 8 connection(s)
I20260618 22:38:06.095942 8271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:38:06.099148 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 8271
I20260618 22:38:06.099346 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:38:06.099890 19034 external_mini_cluster.cc:1428] Setting key 93d25fd7a122d621f31240c7bb64ba3b
I20260618 22:38:06.099994 8338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:06.113979 8338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81: Bootstrap starting.
Jun 18 22:38:06 dist-test-slave-h7jz krb5kdc[8258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:38:06.117265 8338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:06.118377 8338 log.cc:824] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:06.121411 8338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81: No bootstrap required, opened a new log
I20260618 22:38:06.122828 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:06.101658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42135 (local address 127.18.150.190:42403)
0618 22:38:06.102425 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:06.102437 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:06.102480 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:38:06.104536 (+ 2056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:06.104558 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:06.104963 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:06.105293 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:06.106973 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:06.108228 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:06.109902 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:06.110641 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:06.110776 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:06.115356 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:06.115363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:06.115377 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:06.115422 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:06.118351 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:06.119129 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:06.119136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:06.119141 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:06.119316 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:06.119806 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:06.119811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:06.119815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:06.120467 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:06.120776 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:06.121145 (+ 369us) server_negotiation.cc:299] Negotiation successful
0618 22:38:06.121497 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":352,"threads_started":1}
I20260618 22:38:06.126556 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:38:06.129341 8338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923811dee766415b82d5592fb2a9fc81" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42403 } }
I20260618 22:38:06.129696 8338 raft_consensus.cc:374] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:38:06.129792 8338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 923811dee766415b82d5592fb2a9fc81, State: Initialized, Role: FOLLOWER
I20260618 22:38:06.130368 8338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923811dee766415b82d5592fb2a9fc81" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42403 } }
I20260618 22:38:06.130676 8338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:38:06.130822 8338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:38:06.131004 8338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:06.133997 8338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923811dee766415b82d5592fb2a9fc81" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42403 } }
I20260618 22:38:06.134528 8338 leader_election.cc:302] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 923811dee766415b82d5592fb2a9fc81; no voters:
I20260618 22:38:06.135176 8338 leader_election.cc:288] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:38:06.135270 8348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:38:06.136283 8348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [term 1 LEADER]: Becoming Leader. State: Replica: 923811dee766415b82d5592fb2a9fc81, State: Running, Role: LEADER
I20260618 22:38:06.136904 8348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923811dee766415b82d5592fb2a9fc81" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42403 } }
I20260618 22:38:06.137257 8338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:06.139513 8349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "923811dee766415b82d5592fb2a9fc81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923811dee766415b82d5592fb2a9fc81" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42403 } } }
I20260618 22:38:06.139923 8349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:06.140439 8350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 923811dee766415b82d5592fb2a9fc81. Latest consensus state: current_term: 1 leader_uuid: "923811dee766415b82d5592fb2a9fc81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923811dee766415b82d5592fb2a9fc81" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42403 } } }
I20260618 22:38:06.140758 8350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:06.140832 8357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:38:06.147306 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:06.130788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48427 (local address 127.18.150.190:42403)
0618 22:38:06.130963 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:06.130969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:06.131473 (+ 504us) server_negotiation.cc:407] Connection header received
0618 22:38:06.131680 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:06.131684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:06.131747 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:06.131833 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:06.133535 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:06.134346 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:06.136114 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:06.136758 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:06.136895 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:06.139734 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:06.139739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:06.139742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:06.139772 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:06.143338 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:06.144305 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:06.144311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:06.144314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:06.144487 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:06.145042 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:06.145048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:06.145052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:06.145437 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:06.145706 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:06.146763 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0618 22:38:06.146941 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:38:06.152259 8357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:38:06.162053 8357 catalog_manager.cc:1347] Generated new cluster ID: b6a62268ef364b6189631e3a4138039a
I20260618 22:38:06.162182 8357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:38:06.207717 8357 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:38:06.209044 8357 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:38:06.220613 8357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 923811dee766415b82d5592fb2a9fc81: Generated new TSK 0
I20260618 22:38:06.221412 8357 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:38:06.229512 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8271
I20260618 22:38:06.231103 8333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:38:06.322712 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 8271
W20260618 22:38:06.336385 8256 connection.cc:535] client connection to 127.18.150.190:42403 recv error: Network error: failed to read from TLS socket (remote: 127.18.150.190:42403): Connection reset by peer (error 104)
2026-06-18T22:38:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1887 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:38:06 dist-test-slave-h7jz krb5kdc[8370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:38:06 dist-test-slave-h7jz krb5kdc[8370](info): set up 2 sockets
Jun 18 22:38:06 dist-test-slave-h7jz krb5kdc[8370](info): commencing operation
krb5kdc: starting...
W20260618 22:38:07.334084 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:38:07 dist-test-slave-h7jz krb5kdc[8370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:38:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:38:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:38:07.584421 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33939
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:33939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:38:07.911051 8383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:07.911377 8383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:07.911523 8383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:07.919072 8383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:07.919209 8383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:07.919303 8383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:07.919389 8383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:07.919451 8383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:07.929715 8383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:33939
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:07.931073 8383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:07.932451 8383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:07.944125 8391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:07.944094 8389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:07.944362 8388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:07.945137 8383 server_base.cc:1034] running on GCE node
I20260618 22:38:07.945899 8383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:07.947266 8383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:07.948487 8383 hybrid_clock.cc:648] HybridClock initialized: now 1781822287948443 us; error 31 us; skew 500 ppm
I20260618 22:38:07.948813 8383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:07 dist-test-slave-h7jz krb5kdc[8370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822287, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:07.951745 8383 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:38:07.953068 8383 webserver.cc:460] Webserver started at http://127.18.150.190:39471/ using document root <none> and password file <none>
I20260618 22:38:07.953729 8383 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:07.953835 8383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:07.954170 8383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:07.957165 8383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "3f14ec95883c4d1ba665353e928da400"
format_stamp: "Formatted at 2026-06-18 22:38:07 on dist-test-slave-h7jz"
server_key: "a85b322b75c0b5e9dbc603dcf5082150"
server_key_iv: "3e7f129bf70003336d7b0830412a11d7"
server_key_version: "encryptionkey@0"
I20260618 22:38:07.957963 8383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "3f14ec95883c4d1ba665353e928da400"
format_stamp: "Formatted at 2026-06-18 22:38:07 on dist-test-slave-h7jz"
server_key: "a85b322b75c0b5e9dbc603dcf5082150"
server_key_iv: "3e7f129bf70003336d7b0830412a11d7"
server_key_version: "encryptionkey@0"
I20260618 22:38:07.963184 8383 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260618 22:38:07.966928 8398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:07.968372 8383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:38:07.968562 8383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "3f14ec95883c4d1ba665353e928da400"
format_stamp: "Formatted at 2026-06-18 22:38:07 on dist-test-slave-h7jz"
server_key: "a85b322b75c0b5e9dbc603dcf5082150"
server_key_iv: "3e7f129bf70003336d7b0830412a11d7"
server_key_version: "encryptionkey@0"
I20260618 22:38:07.968750 8383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:07.989282 8383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:07.993201 8383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:07.993476 8383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:08.010603 8383 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:33939
I20260618 22:38:08.010609 8449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:33939 every 8 connection(s)
I20260618 22:38:08.011986 8383 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:38:08.015661 8450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:08.016836 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 8383
I20260618 22:38:08.017002 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:38:08.017418 19034 external_mini_cluster.cc:1428] Setting key 827118015fea9fc3f1ec29f6df220b7a
I20260618 22:38:08.028441 8450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400: Bootstrap starting.
I20260618 22:38:08.030237 8453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:08.019234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34957 (local address 127.18.150.190:33939)
0618 22:38:08.020079 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:08.020090 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:08.020133 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:38:08.021701 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:08.021727 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:08.022187 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:08.022521 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:08.023128 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:08.024525 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:08.025840 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:08.026617 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:08.026782 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:08.027993 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:08.028001 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:08.028015 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:08.028545 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:08.028638 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:08.028667 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:08.028798 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:08.029066 (+ 268us) server_negotiation.cc:299] Negotiation successful
0618 22:38:08.029234 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":415,"threads_started":1}
I20260618 22:38:08.031629 8450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:08.032711 8450 log.cc:824] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:08.033838 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:38:08.035465 8450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400: No bootstrap required, opened a new log
I20260618 22:38:08.044540 8453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:08.038028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50413 (local address 127.18.150.190:33939)
0618 22:38:08.038210 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:08.038216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:08.038466 (+ 250us) server_negotiation.cc:407] Connection header received
0618 22:38:08.038630 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:08.038637 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:08.038695 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:08.038774 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:08.039365 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:08.040128 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:08.042104 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:08.042560 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:08.042647 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:08.043875 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:08.043880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:08.043883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:08.043900 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:08.043935 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:08.043950 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:08.044030 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:08.044201 (+ 171us) server_negotiation.cc:299] Negotiation successful
0618 22:38:08.044242 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260618 22:38:08.044615 8450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f14ec95883c4d1ba665353e928da400" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33939 } }
I20260618 22:38:08.045056 8450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:38:08.045176 8450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f14ec95883c4d1ba665353e928da400, State: Initialized, Role: FOLLOWER
I20260618 22:38:08.045853 8450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f14ec95883c4d1ba665353e928da400" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33939 } }
I20260618 22:38:08.046155 8450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:38:08.046298 8450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:38:08.046430 8450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:08.048501 8450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f14ec95883c4d1ba665353e928da400" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33939 } }
I20260618 22:38:08.048944 8450 leader_election.cc:302] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f14ec95883c4d1ba665353e928da400; no voters:
I20260618 22:38:08.049441 8450 leader_election.cc:288] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:38:08.049569 8460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:38:08.050222 8460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [term 1 LEADER]: Becoming Leader. State: Replica: 3f14ec95883c4d1ba665353e928da400, State: Running, Role: LEADER
I20260618 22:38:08.050736 8460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f14ec95883c4d1ba665353e928da400" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33939 } }
I20260618 22:38:08.051453 8450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:08.053331 8461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f14ec95883c4d1ba665353e928da400" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f14ec95883c4d1ba665353e928da400" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33939 } } }
I20260618 22:38:08.053715 8461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:08.053486 8462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f14ec95883c4d1ba665353e928da400. Latest consensus state: current_term: 1 leader_uuid: "3f14ec95883c4d1ba665353e928da400" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f14ec95883c4d1ba665353e928da400" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33939 } } }
I20260618 22:38:08.053911 8462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:08.058775 8467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:38:08.063020 8467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:38:08.071118 8467 catalog_manager.cc:1347] Generated new cluster ID: 1c4222e60cb2401881f28df99aaea845
I20260618 22:38:08.071229 8467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:38:08.085254 8467 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:38:08.086515 8467 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:38:08.095541 8467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f14ec95883c4d1ba665353e928da400: Generated new TSK 0
I20260618 22:38:08.096354 8467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:38:08.102509 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8383
W20260618 22:38:08.104144 8445 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34957
W20260618 22:38:08.104897 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8383: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:38:08.105060 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 8383
W20260618 22:38:08.118484 8368 connection.cc:535] client connection to 127.18.150.190:33939 recv error: Network error: failed to read from TLS socket (remote: 127.18.150.190:33939): Connection reset by peer (error 104)
2026-06-18T22:38:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1782 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:38:08 dist-test-slave-h7jz krb5kdc[8481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:38:08 dist-test-slave-h7jz krb5kdc[8481](info): set up 2 sockets
Jun 18 22:38:08 dist-test-slave-h7jz krb5kdc[8481](info): commencing operation
krb5kdc: starting...
W20260618 22:38:09.114153 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:38:09 dist-test-slave-h7jz krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:38:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:38:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:38:09.351166 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:42497
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:42497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:38:09.668042 8494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:09.668327 8494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:09.668457 8494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:09.675634 8494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:09.675752 8494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:09.675812 8494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:09.675894 8494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:09.675951 8494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:09.685788 8494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:42497
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:42497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:09.687068 8494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:09.688333 8494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:09.699447 8499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:09.699476 8500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:09.699589 8502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:09.700294 8494 server_base.cc:1034] running on GCE node
I20260618 22:38:09.700960 8494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:09.702280 8494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:09.703465 8494 hybrid_clock.cc:648] HybridClock initialized: now 1781822289703424 us; error 39 us; skew 500 ppm
I20260618 22:38:09.703765 8494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:09 dist-test-slave-h7jz krb5kdc[8481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822289, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:09.709794 8494 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:38:09.711210 8494 webserver.cc:460] Webserver started at http://127.18.150.190:41237/ using document root <none> and password file <none>
I20260618 22:38:09.711874 8494 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:09.711988 8494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:09.712270 8494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:09.715251 8494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "c661d489beca4975b0589724c7511cf0"
format_stamp: "Formatted at 2026-06-18 22:38:09 on dist-test-slave-h7jz"
server_key: "f32b2d3b9c7ef7c2b80ceace0bb562f4"
server_key_iv: "37b8bcca0e609f2929e1b1badb2e99e0"
server_key_version: "encryptionkey@0"
I20260618 22:38:09.715999 8494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "c661d489beca4975b0589724c7511cf0"
format_stamp: "Formatted at 2026-06-18 22:38:09 on dist-test-slave-h7jz"
server_key: "f32b2d3b9c7ef7c2b80ceace0bb562f4"
server_key_iv: "37b8bcca0e609f2929e1b1badb2e99e0"
server_key_version: "encryptionkey@0"
I20260618 22:38:09.721210 8494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:38:09.724681 8509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:09.726431 8494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:38:09.726614 8494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "c661d489beca4975b0589724c7511cf0"
format_stamp: "Formatted at 2026-06-18 22:38:09 on dist-test-slave-h7jz"
server_key: "f32b2d3b9c7ef7c2b80ceace0bb562f4"
server_key_iv: "37b8bcca0e609f2929e1b1badb2e99e0"
server_key_version: "encryptionkey@0"
I20260618 22:38:09.726857 8494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:09.734229 8494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:09.738381 8494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:09.738622 8494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:09.756872 8494 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:42497
I20260618 22:38:09.756891 8560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:42497 every 8 connection(s)
I20260618 22:38:09.758246 8494 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:38:09.761852 8561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:09.764590 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 8494
I20260618 22:38:09.764760 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:38:09.765298 19034 external_mini_cluster.cc:1428] Setting key d9010711b654dde89226c0e4219f48de
I20260618 22:38:09.772449 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:09.767023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47993 (local address 127.18.150.190:42497)
0618 22:38:09.767749 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:09.767761 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:09.767801 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:38:09.769373 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:09.769395 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:09.769737 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:09.770103 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:09.771014 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:09.771020 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:09.771036 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:09.771049 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:09.771110 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:09.771131 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:09.771291 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:09.771415 (+ 124us) server_negotiation.cc:299] Negotiation successful
0618 22:38:09.771599 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":332,"threads_started":1}
I20260618 22:38:09.773262 8561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0: Bootstrap starting.
I20260618 22:38:09.775712 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:38:09.776352 8561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:09.777501 8561 log.cc:824] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:09.780525 8561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0: No bootstrap required, opened a new log
I20260618 22:38:09.782622 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:09.780344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55535 (local address 127.18.150.190:42497)
0618 22:38:09.780509 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:09.780519 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:09.780930 (+ 411us) server_negotiation.cc:407] Connection header received
0618 22:38:09.781050 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:09.781055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:09.781115 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:09.781218 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:09.782025 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:09.782030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:09.782035 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:09.782054 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:09.782101 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:09.782124 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:09.782236 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:09.782281 (+ 45us) server_negotiation.cc:299] Negotiation successful
0618 22:38:09.782339 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:38:09.788103 8561 raft_consensus.cc:348] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c661d489beca4975b0589724c7511cf0" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42497 } }
I20260618 22:38:09.788448 8561 raft_consensus.cc:374] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:38:09.788520 8561 raft_consensus.cc:729] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c661d489beca4975b0589724c7511cf0, State: Initialized, Role: FOLLOWER
I20260618 22:38:09.788996 8561 consensus_queue.cc:260] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c661d489beca4975b0589724c7511cf0" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42497 } }
I20260618 22:38:09.789191 8561 raft_consensus.cc:388] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:38:09.789284 8561 raft_consensus.cc:482] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:38:09.789412 8561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:09.791510 8561 raft_consensus.cc:504] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c661d489beca4975b0589724c7511cf0" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42497 } }
I20260618 22:38:09.791934 8561 leader_election.cc:302] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c661d489beca4975b0589724c7511cf0; no voters:
I20260618 22:38:09.792454 8561 leader_election.cc:288] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:38:09.792613 8571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:38:09.793491 8571 raft_consensus.cc:686] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [term 1 LEADER]: Becoming Leader. State: Replica: c661d489beca4975b0589724c7511cf0, State: Running, Role: LEADER
I20260618 22:38:09.793998 8571 consensus_queue.cc:237] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c661d489beca4975b0589724c7511cf0" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42497 } }
I20260618 22:38:09.794435 8561 sys_catalog.cc:564] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:09.796710 8572 sys_catalog.cc:455] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c661d489beca4975b0589724c7511cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c661d489beca4975b0589724c7511cf0" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42497 } } }
I20260618 22:38:09.797124 8572 sys_catalog.cc:458] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:09.798178 8573 sys_catalog.cc:455] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c661d489beca4975b0589724c7511cf0. Latest consensus state: current_term: 1 leader_uuid: "c661d489beca4975b0589724c7511cf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c661d489beca4975b0589724c7511cf0" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 42497 } } }
I20260618 22:38:09.798543 8573 sys_catalog.cc:458] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:09.799029 8580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:38:09.803668 8580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:38:09.811446 8580 catalog_manager.cc:1347] Generated new cluster ID: 17f8005d593040db91df41bd2e6a4db5
I20260618 22:38:09.811558 8580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:38:09.846899 8580 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:38:09.847312 8580 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:38:09.866024 8580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c661d489beca4975b0589724c7511cf0: Generated new TSK 0
I20260618 22:38:09.866827 8580 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:38:09.875196 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8494
W20260618 22:38:09.876237 8556 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47993
W20260618 22:38:09.876873 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8494: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:38:09.877002 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 8494
2026-06-18T22:38:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1771 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:38:09 dist-test-slave-h7jz krb5kdc[8592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:38:09 dist-test-slave-h7jz krb5kdc[8592](info): set up 2 sockets
Jun 18 22:38:09 dist-test-slave-h7jz krb5kdc[8592](info): commencing operation
krb5kdc: starting...
W20260618 22:38:10.878088 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.969s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:38:11 dist-test-slave-h7jz krb5kdc[8592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:38:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:38:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:38:11.111021 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35569
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:35569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:38:11.425554 8605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:11.425820 8605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:11.425909 8605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:11.432941 8605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:11.433037 8605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:11.433084 8605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:11.433140 8605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:11.433174 8605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:11.442723 8605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:35569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:11.443944 8605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:11.445175 8605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:11.456298 8613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:11.456373 8610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:11.456348 8611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:11.457268 8605 server_base.cc:1034] running on GCE node
I20260618 22:38:11.458091 8605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:11.459447 8605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:11.460649 8605 hybrid_clock.cc:648] HybridClock initialized: now 1781822291460594 us; error 48 us; skew 500 ppm
I20260618 22:38:11.460968 8605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:11 dist-test-slave-h7jz krb5kdc[8592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822291, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:11.463836 8605 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:38:11.465207 8605 webserver.cc:460] Webserver started at http://127.18.150.190:38011/ using document root <none> and password file <none>
I20260618 22:38:11.465869 8605 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:11.466003 8605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:11.466288 8605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:11.469082 8605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "3925ffce126b4b9d9d29ea53e7ebe899"
format_stamp: "Formatted at 2026-06-18 22:38:11 on dist-test-slave-h7jz"
server_key: "4da74fd9676d1b1d01028e51afa0d917"
server_key_iv: "cd8d74172bce8195237ddba000d29a3c"
server_key_version: "encryptionkey@0"
I20260618 22:38:11.469798 8605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "3925ffce126b4b9d9d29ea53e7ebe899"
format_stamp: "Formatted at 2026-06-18 22:38:11 on dist-test-slave-h7jz"
server_key: "4da74fd9676d1b1d01028e51afa0d917"
server_key_iv: "cd8d74172bce8195237ddba000d29a3c"
server_key_version: "encryptionkey@0"
I20260618 22:38:11.474725 8605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260618 22:38:11.477888 8620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:11.479137 8605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:38:11.479310 8605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "3925ffce126b4b9d9d29ea53e7ebe899"
format_stamp: "Formatted at 2026-06-18 22:38:11 on dist-test-slave-h7jz"
server_key: "4da74fd9676d1b1d01028e51afa0d917"
server_key_iv: "cd8d74172bce8195237ddba000d29a3c"
server_key_version: "encryptionkey@0"
I20260618 22:38:11.479480 8605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:11.522187 8605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:11.526609 8605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:11.526912 8605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:11.545140 8605 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:35569
I20260618 22:38:11.545140 8671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:35569 every 8 connection(s)
I20260618 22:38:11.546612 8605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:38:11.550387 8672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:11.554086 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 8605
I20260618 22:38:11.554270 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:38:11.554793 19034 external_mini_cluster.cc:1428] Setting key 678d65f34d4731372b28a47b858af33d
I20260618 22:38:11.562932 8672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899: Bootstrap starting.
I20260618 22:38:11.566699 8672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:11.568029 8672 log.cc:824] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:38:11 dist-test-slave-h7jz krb5kdc[8592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:38:11.571182 8672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899: No bootstrap required, opened a new log
I20260618 22:38:11.577030 8675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:11.556551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59131 (local address 127.18.150.190:35569)
0618 22:38:11.557285 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:11.557297 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:11.557339 (+ 42us) server_negotiation.cc:407] Connection header received
0618 22:38:11.558967 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:11.558988 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:11.559403 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:11.559738 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:11.561472 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:11.562809 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:11.564562 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:11.565252 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:11.565376 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:11.570205 (+ 4829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:11.570214 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:11.570228 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:11.570266 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:11.573107 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:11.573927 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:11.573933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:11.573961 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:11.574130 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:11.574495 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:11.574498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:11.574500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:11.575009 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:11.575219 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:11.575651 (+ 432us) server_negotiation.cc:299] Negotiation successful
0618 22:38:11.576019 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":351,"threads_started":1}
I20260618 22:38:11.579849 8672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3925ffce126b4b9d9d29ea53e7ebe899" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35569 } }
I20260618 22:38:11.580240 8672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:38:11.580340 8672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3925ffce126b4b9d9d29ea53e7ebe899, State: Initialized, Role: FOLLOWER
I20260618 22:38:11.580772 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:38:11.580945 8672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3925ffce126b4b9d9d29ea53e7ebe899" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35569 } }
I20260618 22:38:11.581190 8672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:38:11.581306 8672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:38:11.581459 8672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:11.584565 8672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3925ffce126b4b9d9d29ea53e7ebe899" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35569 } }
I20260618 22:38:11.585155 8672 leader_election.cc:302] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3925ffce126b4b9d9d29ea53e7ebe899; no voters:
I20260618 22:38:11.585816 8672 leader_election.cc:288] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:38:11.585987 8681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:38:11.587059 8681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [term 1 LEADER]: Becoming Leader. State: Replica: 3925ffce126b4b9d9d29ea53e7ebe899, State: Running, Role: LEADER
I20260618 22:38:11.587769 8681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3925ffce126b4b9d9d29ea53e7ebe899" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35569 } }
I20260618 22:38:11.588004 8672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:11.591056 8684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3925ffce126b4b9d9d29ea53e7ebe899. Latest consensus state: current_term: 1 leader_uuid: "3925ffce126b4b9d9d29ea53e7ebe899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3925ffce126b4b9d9d29ea53e7ebe899" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35569 } } }
I20260618 22:38:11.591511 8684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:11.592113 8683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3925ffce126b4b9d9d29ea53e7ebe899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3925ffce126b4b9d9d29ea53e7ebe899" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35569 } } }
I20260618 22:38:11.592559 8683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:11.604233 8690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:38:11.610683 8690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:38:11.621160 8675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:11.585898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53537 (local address 127.18.150.190:35569)
0618 22:38:11.586090 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:11.586096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:11.602267 (+ 16171us) server_negotiation.cc:407] Connection header received
0618 22:38:11.602496 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:11.602504 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:11.602608 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:11.602728 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:11.605142 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:11.606162 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:11.608243 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:11.609005 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:11.609157 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:11.612727 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:11.612735 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:11.612740 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:11.612794 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:11.616829 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:11.617817 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:11.617823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:11.617828 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:11.618017 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:11.618513 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:11.618520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:11.618524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:11.619046 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:11.619330 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:11.620521 (+ 1191us) server_negotiation.cc:299] Negotiation successful
0618 22:38:11.620750 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260618 22:38:11.624567 8690 catalog_manager.cc:1347] Generated new cluster ID: 6a7c7af5947d4917a9d1a9a1d25c432e
I20260618 22:38:11.624666 8690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:38:11.642532 8690 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:38:11.644484 8690 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:38:11.657601 8690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3925ffce126b4b9d9d29ea53e7ebe899: Generated new TSK 0
I20260618 22:38:11.658730 8690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:38:11.669239 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8605
I20260618 22:38:11.670542 8667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:38:11.764545 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 8605
2026-06-18T22:38:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1887 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:38:11 dist-test-slave-h7jz krb5kdc[8705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:38:11 dist-test-slave-h7jz krb5kdc[8705](info): set up 2 sockets
Jun 18 22:38:11 dist-test-slave-h7jz krb5kdc[8705](info): commencing operation
krb5kdc: starting...
W20260618 22:38:12.786127 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:38:12 dist-test-slave-h7jz krb5kdc[8705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:38:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:38:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:38:13.019150 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35031
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:42745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:35031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:38:13.329109 8718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:13.329421 8718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:13.329552 8718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:13.336853 8718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:13.336984 8718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:13.337042 8718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:13.337124 8718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:13.337175 8718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:13.347675 8718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:42745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:35031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:35031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:13.349022 8718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:13.350358 8718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:13.361472 8726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:13.361481 8723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:13.361445 8724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:13.362051 8718 server_base.cc:1034] running on GCE node
I20260618 22:38:13.362929 8718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:13.364298 8718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:13.365512 8718 hybrid_clock.cc:648] HybridClock initialized: now 1781822293365467 us; error 41 us; skew 500 ppm
I20260618 22:38:13.365852 8718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:13 dist-test-slave-h7jz krb5kdc[8705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822293, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:13.368791 8718 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:38:13.370263 8718 webserver.cc:460] Webserver started at http://127.18.150.190:35263/ using document root <none> and password file <none>
I20260618 22:38:13.370919 8718 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:13.371034 8718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:13.371309 8718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:13.374311 8718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "3cb1f0481be845eda4a3fbe0d3f177f6"
format_stamp: "Formatted at 2026-06-18 22:38:13 on dist-test-slave-h7jz"
server_key: "c1b3378982d9d44888f1d8ff17f881d4"
server_key_iv: "2cc914c434419b5924e8395a86ff0385"
server_key_version: "encryptionkey@0"
I20260618 22:38:13.375041 8718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "3cb1f0481be845eda4a3fbe0d3f177f6"
format_stamp: "Formatted at 2026-06-18 22:38:13 on dist-test-slave-h7jz"
server_key: "c1b3378982d9d44888f1d8ff17f881d4"
server_key_iv: "2cc914c434419b5924e8395a86ff0385"
server_key_version: "encryptionkey@0"
I20260618 22:38:13.379848 8718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260618 22:38:13.383023 8733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:13.384218 8718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260618 22:38:13.384371 8718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "3cb1f0481be845eda4a3fbe0d3f177f6"
format_stamp: "Formatted at 2026-06-18 22:38:13 on dist-test-slave-h7jz"
server_key: "c1b3378982d9d44888f1d8ff17f881d4"
server_key_iv: "2cc914c434419b5924e8395a86ff0385"
server_key_version: "encryptionkey@0"
I20260618 22:38:13.384541 8718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:13.397475 8718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:13.401290 8718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:13.401535 8718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:13.417881 8718 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:35031
I20260618 22:38:13.417884 8784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:35031 every 8 connection(s)
I20260618 22:38:13.419255 8718 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:38:13.420187 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 8718
I20260618 22:38:13.420360 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:38:13.420773 19034 external_mini_cluster.cc:1428] Setting key eb991da3a8f3fe62a2dbf2d53dd2abfe
I20260618 22:38:13.423405 8785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:13.433573 8788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:13.422608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40973 (local address 127.18.150.190:35031)
0618 22:38:13.423296 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:13.423308 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:13.423367 (+ 59us) server_negotiation.cc:407] Connection header received
0618 22:38:13.424992 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:13.425013 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:13.425389 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:13.425710 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:13.426278 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:13.427565 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:13.429420 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:13.430103 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:13.430238 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:13.431406 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:13.431413 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:13.431426 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:13.431952 (+ 526us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:13.432023 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:13.432046 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:13.432127 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:13.432373 (+ 246us) server_negotiation.cc:299] Negotiation successful
0618 22:38:13.432634 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":299,"threads_started":1}
I20260618 22:38:13.434895 8785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6: Bootstrap starting.
I20260618 22:38:13.436947 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:38:13.437870 8785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:13.438809 8785 log.cc:824] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:13.441448 8785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6: No bootstrap required, opened a new log
I20260618 22:38:13.447129 8788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:13.440835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34395 (local address 127.18.150.190:35031)
0618 22:38:13.441018 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:13.441024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:13.441480 (+ 456us) server_negotiation.cc:407] Connection header received
0618 22:38:13.441647 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:13.441651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:13.441706 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:13.441779 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:13.442388 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:13.443072 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:13.444828 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:13.445283 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:13.445391 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:13.446458 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:13.446463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:13.446466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:13.446486 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:13.446522 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:13.446539 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:13.446613 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:13.446747 (+ 134us) server_negotiation.cc:299] Negotiation successful
0618 22:38:13.446820 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:38:13.450446 8785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb1f0481be845eda4a3fbe0d3f177f6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35031 } }
I20260618 22:38:13.450803 8785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:38:13.450873 8785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb1f0481be845eda4a3fbe0d3f177f6, State: Initialized, Role: FOLLOWER
I20260618 22:38:13.451395 8785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb1f0481be845eda4a3fbe0d3f177f6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35031 } }
I20260618 22:38:13.451601 8785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:38:13.451694 8785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:38:13.451812 8785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:13.453835 8785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb1f0481be845eda4a3fbe0d3f177f6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35031 } }
I20260618 22:38:13.454381 8785 leader_election.cc:302] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cb1f0481be845eda4a3fbe0d3f177f6; no voters:
I20260618 22:38:13.454879 8785 leader_election.cc:288] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:38:13.455003 8795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:38:13.455574 8795 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [term 1 LEADER]: Becoming Leader. State: Replica: 3cb1f0481be845eda4a3fbe0d3f177f6, State: Running, Role: LEADER
I20260618 22:38:13.456077 8795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb1f0481be845eda4a3fbe0d3f177f6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35031 } }
I20260618 22:38:13.456766 8785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:13.458961 8796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cb1f0481be845eda4a3fbe0d3f177f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb1f0481be845eda4a3fbe0d3f177f6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35031 } } }
I20260618 22:38:13.458807 8797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cb1f0481be845eda4a3fbe0d3f177f6. Latest consensus state: current_term: 1 leader_uuid: "3cb1f0481be845eda4a3fbe0d3f177f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb1f0481be845eda4a3fbe0d3f177f6" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 35031 } } }
I20260618 22:38:13.459451 8796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:13.459494 8797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:13.462790 8803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:38:13.467638 8803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:38:13.477366 8803 catalog_manager.cc:1347] Generated new cluster ID: c344d95b338e4abc85dbbd99cd897d4e
I20260618 22:38:13.477504 8803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:38:13.488523 8803 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:38:13.489909 8803 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:38:13.498932 8803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cb1f0481be845eda4a3fbe0d3f177f6: Generated new TSK 0
I20260618 22:38:13.499812 8803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:38:13.510561 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8718
W20260618 22:38:13.511690 8780 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40973
W20260618 22:38:13.512343 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8718: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:38:13.512475 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 8718
2026-06-18T22:38:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1746 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:38:13 dist-test-slave-h7jz krb5kdc[8816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:38:13 dist-test-slave-h7jz krb5kdc[8816](info): set up 2 sockets
Jun 18 22:38:13 dist-test-slave-h7jz krb5kdc[8816](info): commencing operation
krb5kdc: starting...
W20260618 22:38:14.518118 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:38:14 dist-test-slave-h7jz krb5kdc[8816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:38:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:38:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:38:14.754482 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33559
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:44597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:33559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:38:15.075212 8829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:15.075528 8829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:15.075677 8829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:15.083056 8829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:15.083194 8829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:15.083295 8829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:15.083395 8829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:15.083457 8829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:15.093677 8829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:44597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:33559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:15.095018 8829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:15.096374 8829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:15.108177 8835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:15.108213 8834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:15.108268 8837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:15.108834 8829 server_base.cc:1034] running on GCE node
I20260618 22:38:15.109658 8829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:15.111215 8829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:15.112452 8829 hybrid_clock.cc:648] HybridClock initialized: now 1781822295112388 us; error 45 us; skew 500 ppm
I20260618 22:38:15.112838 8829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:15 dist-test-slave-h7jz krb5kdc[8816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822295, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:15.119659 8829 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:38:15.121099 8829 webserver.cc:460] Webserver started at http://127.18.150.190:35683/ using document root <none> and password file <none>
I20260618 22:38:15.121759 8829 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:15.121860 8829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:15.122146 8829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:15.125058 8829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "20b1dd9795fc4ab8971469f2316c05ff"
format_stamp: "Formatted at 2026-06-18 22:38:15 on dist-test-slave-h7jz"
server_key: "7209a7514ad1338254a6c87068b54492"
server_key_iv: "2cbfedd35c5cace43e86ca2293631e1b"
server_key_version: "encryptionkey@0"
I20260618 22:38:15.125763 8829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "20b1dd9795fc4ab8971469f2316c05ff"
format_stamp: "Formatted at 2026-06-18 22:38:15 on dist-test-slave-h7jz"
server_key: "7209a7514ad1338254a6c87068b54492"
server_key_iv: "2cbfedd35c5cace43e86ca2293631e1b"
server_key_version: "encryptionkey@0"
I20260618 22:38:15.130945 8829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260618 22:38:15.134301 8844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:15.135659 8829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:38:15.135800 8829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "20b1dd9795fc4ab8971469f2316c05ff"
format_stamp: "Formatted at 2026-06-18 22:38:15 on dist-test-slave-h7jz"
server_key: "7209a7514ad1338254a6c87068b54492"
server_key_iv: "2cbfedd35c5cace43e86ca2293631e1b"
server_key_version: "encryptionkey@0"
I20260618 22:38:15.135998 8829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:15.141527 8829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:15.145313 8829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:15.145587 8829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:15.161919 8829 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:33559
I20260618 22:38:15.161964 8895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:33559 every 8 connection(s)
I20260618 22:38:15.163257 8829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:38:15.166939 8896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:15.168320 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 8829
I20260618 22:38:15.168465 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:38:15.168880 19034 external_mini_cluster.cc:1428] Setting key 58238d7b60fb19a87e8ce25a429f6eb8
I20260618 22:38:15.177042 8899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:15.170646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35657 (local address 127.18.150.190:33559)
0618 22:38:15.171470 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:15.171486 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:15.171540 (+ 54us) server_negotiation.cc:407] Connection header received
0618 22:38:15.173284 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:15.173321 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:15.173797 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:15.174286 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:15.175194 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:15.175203 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:15.175219 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:15.175234 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:15.175299 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:15.175328 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:15.175525 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:15.175651 (+ 126us) server_negotiation.cc:299] Negotiation successful
0618 22:38:15.175828 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":330,"threads_started":1}
I20260618 22:38:15.180938 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:38:15.181068 8896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff: Bootstrap starting.
I20260618 22:38:15.184135 8896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:15.185256 8896 log.cc:824] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:15.187579 8899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:15.185207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43437 (local address 127.18.150.190:33559)
0618 22:38:15.185439 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:15.185445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:15.185685 (+ 240us) server_negotiation.cc:407] Connection header received
0618 22:38:15.185865 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:15.185870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:15.185963 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:15.186057 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:15.187010 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:15.187016 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:15.187020 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:38:15.187038 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:15.187082 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:38:15.187100 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:15.187205 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:15.187271 (+ 66us) server_negotiation.cc:299] Negotiation successful
0618 22:38:15.187324 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260618 22:38:15.188367 8896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff: No bootstrap required, opened a new log
I20260618 22:38:15.195884 8896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b1dd9795fc4ab8971469f2316c05ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33559 } }
I20260618 22:38:15.196249 8896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:38:15.196343 8896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b1dd9795fc4ab8971469f2316c05ff, State: Initialized, Role: FOLLOWER
I20260618 22:38:15.196936 8896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b1dd9795fc4ab8971469f2316c05ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33559 } }
I20260618 22:38:15.197146 8896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:38:15.197273 8896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:38:15.197448 8896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:15.199903 8896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b1dd9795fc4ab8971469f2316c05ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33559 } }
I20260618 22:38:15.200366 8896 leader_election.cc:302] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20b1dd9795fc4ab8971469f2316c05ff; no voters:
I20260618 22:38:15.200899 8896 leader_election.cc:288] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:38:15.201056 8906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:38:15.201715 8906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [term 1 LEADER]: Becoming Leader. State: Replica: 20b1dd9795fc4ab8971469f2316c05ff, State: Running, Role: LEADER
I20260618 22:38:15.202278 8906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b1dd9795fc4ab8971469f2316c05ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33559 } }
I20260618 22:38:15.202911 8896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:15.204919 8907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20b1dd9795fc4ab8971469f2316c05ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b1dd9795fc4ab8971469f2316c05ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33559 } } }
I20260618 22:38:15.205379 8907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:15.205318 8908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20b1dd9795fc4ab8971469f2316c05ff. Latest consensus state: current_term: 1 leader_uuid: "20b1dd9795fc4ab8971469f2316c05ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20b1dd9795fc4ab8971469f2316c05ff" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33559 } } }
I20260618 22:38:15.205669 8908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:15.206453 8914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:38:15.212070 8914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:38:15.220938 8914 catalog_manager.cc:1347] Generated new cluster ID: 88a891ca9dfd4900bbdb7b7e85c46410
I20260618 22:38:15.221063 8914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:38:15.232892 8914 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:38:15.233385 8914 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:38:15.251785 8914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20b1dd9795fc4ab8971469f2316c05ff: Generated new TSK 0
I20260618 22:38:15.252938 8914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:38:15.263863 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8829
W20260618 22:38:15.264899 8891 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35657
W20260618 22:38:15.265573 19034 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8829: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260618 22:38:15.265697 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 8829
2026-06-18T22:38:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1753 ms)
[----------] 18 tests from AuthTokenIssuingTest (32375 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:38:15 dist-test-slave-h7jz krb5kdc[8927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:38:15 dist-test-slave-h7jz krb5kdc[8927](info): set up 2 sockets
Jun 18 22:38:15 dist-test-slave-h7jz krb5kdc[8927](info): commencing operation
krb5kdc: starting...
W20260618 22:38:16.270262 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:38:16 dist-test-slave-h7jz krb5kdc[8927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:38:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:38:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:38:16.506390 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33427
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:38497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:33427,127.18.150.189:34151,127.18.150.188:41373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:38:16.827441 8940 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:38:16.827786 8940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:16.827893 8940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:16.827989 8940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:16.835311 8940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:16.835417 8940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:16.835480 8940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:16.835551 8940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:16.835633 8940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:16.845456 8940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:38497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:33427,127.18.150.189:34151,127.18.150.188:41373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:16.846766 8940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:16.848079 8940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:16.859208 8945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:16.859233 8946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:16.859344 8948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:16.860035 8940 server_base.cc:1034] running on GCE node
I20260618 22:38:16.860790 8940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:16.862154 8940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:16.863409 8940 hybrid_clock.cc:648] HybridClock initialized: now 1781822296863341 us; error 36 us; skew 500 ppm
I20260618 22:38:16.863725 8940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:16 dist-test-slave-h7jz krb5kdc[8927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822296, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:16.866732 8940 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:38:16.868058 8940 webserver.cc:460] Webserver started at http://127.18.150.190:45407/ using document root <none> and password file <none>
I20260618 22:38:16.868722 8940 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:16.868804 8940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:16.869098 8940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:16.872138 8940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72"
format_stamp: "Formatted at 2026-06-18 22:38:16 on dist-test-slave-h7jz"
server_key: "5d2e229323ecf13766c8b9d9c843d837"
server_key_iv: "e9ae29c1ededff5ddcca4886bab347c2"
server_key_version: "encryptionkey@0"
I20260618 22:38:16.872892 8940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72"
format_stamp: "Formatted at 2026-06-18 22:38:16 on dist-test-slave-h7jz"
server_key: "5d2e229323ecf13766c8b9d9c843d837"
server_key_iv: "e9ae29c1ededff5ddcca4886bab347c2"
server_key_version: "encryptionkey@0"
I20260618 22:38:16.877849 8940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260618 22:38:16.881312 8955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:16.882696 8940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:38:16.882849 8940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72"
format_stamp: "Formatted at 2026-06-18 22:38:16 on dist-test-slave-h7jz"
server_key: "5d2e229323ecf13766c8b9d9c843d837"
server_key_iv: "e9ae29c1ededff5ddcca4886bab347c2"
server_key_version: "encryptionkey@0"
I20260618 22:38:16.883051 8940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:16.903522 8940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:16.907721 8940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:16.907979 8940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:16.924650 8940 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:33427
I20260618 22:38:16.924669 9006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:33427 every 8 connection(s)
I20260618 22:38:16.925928 8940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:38:16.928934 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 8940
I20260618 22:38:16.929142 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:38:16.929797 9007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:16.929981 19034 external_mini_cluster.cc:1428] Setting key 770408b909c6db1d4ce293f3e269f21d
I20260618 22:38:16.938053 9007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:38:16 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822296, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.150.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.189@KRBTEST.COM" created.
I20260618 22:38:16.960527 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:16.941620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46556 (local address 127.18.150.190:33427)
0618 22:38:16.943197 (+ 1577us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:16.943209 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:16.943267 (+ 58us) server_negotiation.cc:407] Connection header received
0618 22:38:16.944685 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:16.944692 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:16.944950 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:16.945158 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:16.946252 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:16.947864 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:16.949188 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:16.949971 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:16.950132 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:16.953334 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:16.953345 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:16.953361 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:16.953402 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:16.956711 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:16.957304 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:16.957311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:16.957315 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:16.957468 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:16.957800 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:16.957803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:16.957805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:16.958319 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:16.958497 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:16.959058 (+ 561us) server_negotiation.cc:299] Negotiation successful
0618 22:38:16.959541 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":220,"server-negotiator.queue_time_us":1385,"spinlock_wait_cycles":37248,"thread_start_us":1208,"threads_started":1}
I20260618 22:38:16.960552 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:16.941623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33427 (local address 127.0.0.1:46556)
0618 22:38:16.942491 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:16.942545 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:16.944332 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:16.945166 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:16.945170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:16.945683 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:16.946055 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:16.946073 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:16.948011 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:16.948015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:16.949047 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:16.949058 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:16.950153 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:16.950158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:16.950283 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:16.950858 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:16.950882 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:16.953105 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:16.956821 (+ 3716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:16.956830 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:16.956839 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:16.957192 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:16.957561 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:16.957565 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:16.957567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:16.957706 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:16.958470 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:16.958476 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:16.958665 (+ 189us) client_negotiation.cc:769] Sending connection context
0618 22:38:16.958898 (+ 233us) client_negotiation.cc:240] Negotiation successful
0618 22:38:16.959363 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":362,"threads_started":1}
I20260618 22:38:16.963478 9007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:38:16.964576 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:16.964163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42676)
0618 22:38:16.964332 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:16.964450 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260618 22:38:16.964843 8957 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
I20260618 22:38:16.966365 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:16.965976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42680)
0618 22:38:16.966137 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:16.966221 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260618 22:38:16.966538 9007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:34151: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab.
Entry for principal kudu/127.18.150.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab.
I20260618 22:38:16.987051 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.189:34151
--webserver_interface=127.18.150.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab
--principal=kudu/127.18.150.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:38497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:33427,127.18.150.189:34151,127.18.150.188:41373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:38:17.015878 9007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } attempt: 1
I20260618 22:38:17.017109 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.016590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42690)
0618 22:38:17.016799 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.016937 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260618 22:38:17.018137 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.017733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42706)
0618 22:38:17.017875 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.017980 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260618 22:38:17.018361 9007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:34151: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
I20260618 22:38:17.086622 9007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } attempt: 2
I20260618 22:38:17.087896 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.087302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42720)
0618 22:38:17.087578 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.087705 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260618 22:38:17.088948 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.088515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42728)
0618 22:38:17.088674 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.088778 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260618 22:38:17.089169 9007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:34151: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
I20260618 22:38:17.180449 9007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } attempt: 3
I20260618 22:38:17.181731 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.181195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42730)
0618 22:38:17.181447 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.181567 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260618 22:38:17.182868 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.182401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42740)
0618 22:38:17.182574 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.182670 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260618 22:38:17.183092 9007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:34151: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
W20260618 22:38:17.301303 9014 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:38:17.301631 9014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:17.301702 9014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:17.301784 9014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:17.309059 9014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:17.309151 9014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:17.309207 9014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:17.309255 9014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:17.309325 9014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:17.319229 9014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:38497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:33427,127.18.150.189:34151,127.18.150.188:41373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab
--principal=kudu/127.18.150.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.189:34151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.150.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:17.320466 9014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:17.321689 9014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:38:17.326349 9007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } attempt: 4
I20260618 22:38:17.327538 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.327040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42748)
0618 22:38:17.327324 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.327410 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260618 22:38:17.328598 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.328256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42756)
0618 22:38:17.328400 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.328479 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260618 22:38:17.328763 9007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:34151: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34151: connect: Connection refused (error 111)
W20260618 22:38:17.333519 9020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:17.333552 9022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:17.333575 9019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:17.334187 9014 server_base.cc:1034] running on GCE node
I20260618 22:38:17.334949 9014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:17.336419 9014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:17.337642 9014 hybrid_clock.cc:648] HybridClock initialized: now 1781822297337575 us; error 41 us; skew 500 ppm
I20260618 22:38:17.338002 9014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:17 dist-test-slave-h7jz krb5kdc[8927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822297, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:17.343766 9014 init.cc:377] Logged in from keytab as kudu/127.18.150.189@KRBTEST.COM (short username kudu)
I20260618 22:38:17.345188 9014 webserver.cc:460] Webserver started at http://127.18.150.189:44189/ using document root <none> and password file <none>
I20260618 22:38:17.345911 9014 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:17.346043 9014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:17.346351 9014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:17.349296 9014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/data/instance:
uuid: "648af6a3b9a3477fb8487a4f0171217b"
format_stamp: "Formatted at 2026-06-18 22:38:17 on dist-test-slave-h7jz"
server_key: "1bd08e1f0794d23b78829c8971cf531f"
server_key_iv: "9fe5a4b411b380c95bcd99c2f77a2c66"
server_key_version: "encryptionkey@0"
I20260618 22:38:17.350090 9014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/wal/instance:
uuid: "648af6a3b9a3477fb8487a4f0171217b"
format_stamp: "Formatted at 2026-06-18 22:38:17 on dist-test-slave-h7jz"
server_key: "1bd08e1f0794d23b78829c8971cf531f"
server_key_iv: "9fe5a4b411b380c95bcd99c2f77a2c66"
server_key_version: "encryptionkey@0"
I20260618 22:38:17.355014 9014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260618 22:38:17.358327 9029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:17.359618 9014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:38:17.359774 9014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/wal
uuid: "648af6a3b9a3477fb8487a4f0171217b"
format_stamp: "Formatted at 2026-06-18 22:38:17 on dist-test-slave-h7jz"
server_key: "1bd08e1f0794d23b78829c8971cf531f"
server_key_iv: "9fe5a4b411b380c95bcd99c2f77a2c66"
server_key_version: "encryptionkey@0"
I20260618 22:38:17.359965 9014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:17.376120 9014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:17.379817 9014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:17.380043 9014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:17.396636 9014 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.189:34151
I20260618 22:38:17.396680 9080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.189:34151 every 8 connection(s)
I20260618 22:38:17.397897 9014 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
I20260618 22:38:17.399619 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 9014
I20260618 22:38:17.399802 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-1/wal/instance
I20260618 22:38:17.400331 19034 external_mini_cluster.cc:1428] Setting key 31faa4352dbef81152a8b6a35be57935
I20260618 22:38:17.402206 9081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:17.410533 9081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:38:17 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822297, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.150.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.188@KRBTEST.COM" created.
I20260618 22:38:17.429517 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.413252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.18.150.190:33427)
0618 22:38:17.413433 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:17.413438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:17.414831 (+ 1393us) server_negotiation.cc:407] Connection header received
0618 22:38:17.416662 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:17.416667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:17.416746 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:17.416831 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:17.418339 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.419018 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.420476 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.420878 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.420996 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.424155 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:17.424159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:17.424162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:17.424195 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:17.426661 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.427368 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.427372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.427374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.427492 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.427874 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.427877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.427879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.428162 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:17.428291 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:17.428981 (+ 690us) server_negotiation.cc:299] Negotiation successful
0618 22:38:17.429203 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:38:17.430503 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.413880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33427 (local address 127.0.0.1:46564)
0618 22:38:17.414675 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.414716 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:17.416389 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:17.417064 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:17.417072 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:17.417577 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:17.418169 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.418186 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.419166 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.419169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.420335 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.420347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.421089 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.421095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.421306 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.421908 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:17.421931 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:17.423952 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:17.426787 (+ 2835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.426796 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.426812 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.427233 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.427590 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.427595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.427599 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.427767 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.428279 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:17.428287 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:17.428619 (+ 332us) client_negotiation.cc:769] Sending connection context
0618 22:38:17.428839 (+ 220us) client_negotiation.cc:240] Negotiation successful
0618 22:38:17.429338 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":320,"threads_started":1}
I20260618 22:38:17.432150 9081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:38:17 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822297, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
I20260618 22:38:17.448812 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.432889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42760)
0618 22:38:17.433038 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.433057 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:17.433149 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:17.434256 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:17.434261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:17.434734 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:17.434899 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.434910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.436228 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.436233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.437404 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.437428 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.438280 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.438286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.438466 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.439002 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:17.439022 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:17.441245 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:17.445591 (+ 4346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.445596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.445600 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.446102 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.446505 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.446509 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.446512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.446829 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.447683 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:17.447687 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:17.447831 (+ 144us) client_negotiation.cc:769] Sending connection context
0618 22:38:17.447965 (+ 134us) client_negotiation.cc:240] Negotiation successful
0618 22:38:17.448407 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260618 22:38:17.449985 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.433137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.18.150.189:34151)
0618 22:38:17.433778 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:17.433786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:17.433817 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:38:17.433879 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:17.433886 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:17.434112 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:17.434223 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:17.435062 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.436100 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.437533 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.438151 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.438241 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.441446 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:17.441457 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:17.441471 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:17.441503 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:17.445447 (+ 3944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.446239 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.446247 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.446250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.446384 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.446949 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.446954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.446957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.447425 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:17.447618 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:17.449177 (+ 1559us) server_negotiation.cc:299] Negotiation successful
0618 22:38:17.449586 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":324,"threads_started":1}
I20260618 22:38:17.452767 9081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:38:17.453827 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.453331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:42964)
0618 22:38:17.453503 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.453652 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260618 22:38:17.454101 9030 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
I20260618 22:38:17.455902 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.455518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:42978)
0618 22:38:17.455687 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.455781 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260618 22:38:17.456213 9081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:41373: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab.
Entry for principal kudu/127.18.150.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab.
I20260618 22:38:17.459112 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.188:41373
--webserver_interface=127.18.150.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab
--principal=kudu/127.18.150.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:38497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:33427,127.18.150.189:34151,127.18.150.188:41373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:38:17.505537 9081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } attempt: 1
I20260618 22:38:17.506714 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.506252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:42980)
0618 22:38:17.506468 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.506568 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260618 22:38:17.507661 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.507301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:42994)
0618 22:38:17.507463 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.507557 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260618 22:38:17.507848 9081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:41373: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
I20260618 22:38:17.576038 9081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } attempt: 2
I20260618 22:38:17.577183 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.576726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43010)
0618 22:38:17.576916 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.577010 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260618 22:38:17.578241 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.577880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43020)
0618 22:38:17.578049 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.578122 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260618 22:38:17.578456 9081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:41373: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
I20260618 22:38:17.628013 9007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } attempt: 5
Jun 18 22:38:17 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822296, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
I20260618 22:38:17.640513 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.628892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.18.150.189:34151)
0618 22:38:17.629063 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:17.629069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:17.629090 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:38:17.629254 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:17.629260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:17.629336 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:17.629447 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:17.630220 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.630916 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.632071 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.632498 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.632603 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.635776 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:17.635780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:17.635784 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:17.635815 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:17.638227 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.638797 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.638802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.638805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.638916 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.639239 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.639244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.639247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.639514 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:17.639638 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:17.639998 (+ 360us) server_negotiation.cc:299] Negotiation successful
0618 22:38:17.640202 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:38:17.640513 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.628736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42762)
0618 22:38:17.628965 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.628987 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:17.629121 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:17.629472 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:17.629476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:17.629858 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:17.630077 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.630086 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.631045 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.631049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.631939 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.631951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.632631 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.632636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.632840 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.633438 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:17.633454 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:17.635602 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:17.638346 (+ 2744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.638352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.638356 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.638697 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.639013 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.639018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.639022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.639146 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.639620 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:17.639625 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:17.639768 (+ 143us) client_negotiation.cc:769] Sending connection context
0618 22:38:17.639888 (+ 120us) client_negotiation.cc:240] Negotiation successful
0618 22:38:17.640160 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260618 22:38:17.641737 9007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:38:17.642831 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.642405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43032)
0618 22:38:17.642578 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.642673 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260618 22:38:17.643736 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.643368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43034)
0618 22:38:17.643510 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.643591 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260618 22:38:17.643971 9007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:41373: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
I20260618 22:38:17.669641 9081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } attempt: 3
I20260618 22:38:17.670818 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.670344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43038)
0618 22:38:17.670564 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.670663 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260618 22:38:17.671998 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.671549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43046)
0618 22:38:17.671798 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.671885 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260618 22:38:17.672210 9081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:41373: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
I20260618 22:38:17.695210 9007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } attempt: 1
I20260618 22:38:17.696450 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.695868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43054)
0618 22:38:17.696075 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.696193 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260618 22:38:17.697381 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.697026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43062)
0618 22:38:17.697170 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.697255 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260618 22:38:17.697579 9007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:41373: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
I20260618 22:38:17.765786 9007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } attempt: 2
I20260618 22:38:17.766949 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.766483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43066)
0618 22:38:17.766668 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.766772 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260618 22:38:17.768041 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.767689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43074)
0618 22:38:17.767834 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.767912 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260618 22:38:17.768301 9007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:41373: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
W20260618 22:38:17.788951 9088 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:38:17.789284 9088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:17.789377 9088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:17.789485 9088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:17.796617 9088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:17.796715 9088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:17.796800 9088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:17.796870 9088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:17.796952 9088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:17.806860 9088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:38497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:33427,127.18.150.189:34151,127.18.150.188:41373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab
--principal=kudu/127.18.150.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.188:41373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.150.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:17.808199 9088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:17.809476 9088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:38:17.815426 9081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } attempt: 4
I20260618 22:38:17.816589 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.816097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43078)
0618 22:38:17.816312 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.816434 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260618 22:38:17.817497 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.817168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43088)
0618 22:38:17.817315 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.817389 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260618 22:38:17.817693 9081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:41373: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
W20260618 22:38:17.821024 9096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:17.821046 9094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:17.821008 9093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:17.821797 9088 server_base.cc:1034] running on GCE node
I20260618 22:38:17.822567 9088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:17.823994 9088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:17.825219 9088 hybrid_clock.cc:648] HybridClock initialized: now 1781822297825142 us; error 49 us; skew 500 ppm
I20260618 22:38:17.825578 9088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:17 dist-test-slave-h7jz krb5kdc[8927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822297, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:17.828598 9088 init.cc:377] Logged in from keytab as kudu/127.18.150.188@KRBTEST.COM (short username kudu)
I20260618 22:38:17.830080 9088 webserver.cc:460] Webserver started at http://127.18.150.188:44877/ using document root <none> and password file <none>
I20260618 22:38:17.830734 9088 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:17.830818 9088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:17.831111 9088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:17.834039 9088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/data/instance:
uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0"
format_stamp: "Formatted at 2026-06-18 22:38:17 on dist-test-slave-h7jz"
server_key: "c49fb606fc14b188507f0d1518d923e1"
server_key_iv: "f29803220a70d16d7321a66c79c8dec2"
server_key_version: "encryptionkey@0"
I20260618 22:38:17.834816 9088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/wal/instance:
uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0"
format_stamp: "Formatted at 2026-06-18 22:38:17 on dist-test-slave-h7jz"
server_key: "c49fb606fc14b188507f0d1518d923e1"
server_key_iv: "f29803220a70d16d7321a66c79c8dec2"
server_key_version: "encryptionkey@0"
I20260618 22:38:17.839879 9088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260618 22:38:17.843187 9103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:17.844512 9088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:38:17.844662 9088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/wal
uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0"
format_stamp: "Formatted at 2026-06-18 22:38:17 on dist-test-slave-h7jz"
server_key: "c49fb606fc14b188507f0d1518d923e1"
server_key_iv: "f29803220a70d16d7321a66c79c8dec2"
server_key_version: "encryptionkey@0"
I20260618 22:38:17.844862 9088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:17.858669 9088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:17.862710 9088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:17.862969 9088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:17.874507 9007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } attempt: 3
I20260618 22:38:17.875653 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.875101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43094)
0618 22:38:17.875343 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.875458 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260618 22:38:17.876715 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.876273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43110)
0618 22:38:17.876484 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.876584 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260618 22:38:17.876983 9007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:41373: Network error: Client connection negotiation failed: client connection to 127.18.150.188:41373: connect: Connection refused (error 111)
I20260618 22:38:17.880000 9088 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.188:41373
I20260618 22:38:17.880017 9154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.188:41373 every 8 connection(s)
I20260618 22:38:17.881292 9088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
I20260618 22:38:17.884908 9155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:17.886894 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 9088
I20260618 22:38:17.887050 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821960891835-19034-0/minicluster-data/master-2/wal/instance
I20260618 22:38:17.887461 19034 external_mini_cluster.cc:1428] Setting key eeb59c2cd63e9ba27a55273f32f309cb
I20260618 22:38:17.893087 9155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:38:17 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
Jun 18 22:38:17 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822297, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:38:17.910509 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.895873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46568 (local address 127.18.150.190:33427)
0618 22:38:17.896067 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:17.896073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:17.896662 (+ 589us) server_negotiation.cc:407] Connection header received
0618 22:38:17.896821 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:17.896826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:17.896906 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:17.897014 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:17.898110 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.899104 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.900870 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.901284 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.901377 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.905158 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:17.905162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:17.905165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:17.905200 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:17.907823 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.908553 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.908558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.908562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.908684 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.909015 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.909021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.909025 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.909283 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:17.909419 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:17.909888 (+ 469us) server_negotiation.cc:299] Negotiation successful
0618 22:38:17.910139 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:38:17.911198 9159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.895807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33427 (local address 127.0.0.1:46568)
0618 22:38:17.896504 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.896544 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:17.896700 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:17.897079 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:17.897086 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:17.897667 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:17.897925 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.897935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.899277 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.899282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.900729 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.900744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.901497 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.901504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.901685 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.902316 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:17.902348 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:17.904945 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:17.907947 (+ 3002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.907958 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.907966 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.908444 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.908785 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.908789 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.908797 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.908920 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.909424 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:17.909433 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:17.909607 (+ 174us) client_negotiation.cc:769] Sending connection context
0618 22:38:17.909919 (+ 312us) client_negotiation.cc:240] Negotiation successful
0618 22:38:17.910274 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":301,"threads_started":1}
I20260618 22:38:17.911216 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.889223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43122 (local address 127.18.150.188:41373)
0618 22:38:17.890009 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:17.890025 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:17.890076 (+ 51us) server_negotiation.cc:407] Connection header received
0618 22:38:17.892320 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:17.892348 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:17.892841 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:17.893287 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:17.894628 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.896294 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.897894 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.898873 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.899042 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.903877 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:17.903889 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:17.903905 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:17.903944 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:17.906915 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.907716 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.907722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.907729 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.907868 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.908269 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.908274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.908278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.908838 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:17.909115 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:17.909594 (+ 479us) server_negotiation.cc:299] Negotiation successful
0618 22:38:17.910034 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":282,"threads_started":1}
I20260618 22:38:17.913473 9155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:38:17 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822297, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
Jun 18 22:38:17 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:38:17.926784 9159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.914219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34151 (local address 127.0.0.1:42768)
0618 22:38:17.914375 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.914390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:17.914481 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:17.914794 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:17.914798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:17.915156 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:17.915285 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.915292 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.916192 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.916197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.917053 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.917064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.917725 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.917730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.917894 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.918457 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:17.918473 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:17.920644 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:17.924078 (+ 3434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.924083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.924086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.924606 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.925030 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.925035 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.925039 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.925211 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.925790 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:17.925795 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:17.926000 (+ 205us) client_negotiation.cc:769] Sending connection context
0618 22:38:17.926148 (+ 148us) client_negotiation.cc:240] Negotiation successful
0618 22:38:17.926423 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260618 22:38:17.926836 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.914267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.18.150.189:34151)
0618 22:38:17.914437 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:17.914441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:17.914457 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:38:17.914608 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:17.914614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:17.914679 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:17.914763 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:17.915413 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.916072 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.917181 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.917592 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.917697 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.920828 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:17.920833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:17.920836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:17.920869 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:17.923935 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.924730 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.924736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.924740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.924891 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.925325 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.925331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.925334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.925666 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:17.925824 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:17.926263 (+ 439us) server_negotiation.cc:299] Negotiation successful
0618 22:38:17.926483 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:38:17.928251 9155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:38:17.929003 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.914095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46578 (local address 127.18.150.190:33427)
0618 22:38:17.914562 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:17.914567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:17.914578 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:38:17.914615 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:17.914618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:17.914670 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:17.914760 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:17.915889 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.916612 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.917786 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.918220 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.918324 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.922590 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:17.922597 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:17.922602 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:17.922641 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:17.925702 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.926484 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.926488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.926491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.926653 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.927075 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.927080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.927083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.927393 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:17.927661 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:17.927916 (+ 255us) server_negotiation.cc:299] Negotiation successful
0618 22:38:17.928113 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382}
Jun 18 22:38:17 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822297, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
Jun 18 22:38:17 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
I20260618 22:38:17.942778 9159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.928971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43124)
0618 22:38:17.929133 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:17.929150 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:17.929241 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:17.929905 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:17.929909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:17.930351 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:17.930517 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.930528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.931796 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.931801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.933161 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:17.933174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.934212 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.934216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:17.934394 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.934991 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:17.935007 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:17.937166 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:17.940330 (+ 3164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.940337 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.940341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.940705 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.941055 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:17.941060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:17.941065 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:17.941193 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:17.941693 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:17.941698 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:17.941848 (+ 150us) client_negotiation.cc:769] Sending connection context
0618 22:38:17.942024 (+ 176us) client_negotiation.cc:240] Negotiation successful
0618 22:38:17.942312 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260618 22:38:17.942819 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.929171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43124 (local address 127.18.150.188:41373)
0618 22:38:17.929542 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:17.929547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:17.929564 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:38:17.929606 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:17.929611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:17.929674 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:17.929753 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:17.930655 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.931660 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.933291 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.933897 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.934025 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.937364 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:17.937369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:17.937372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:17.937402 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:17.940193 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.940829 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.940835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.940838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.940948 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.941294 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.941299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.941304 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.941581 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:17.941715 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:17.942124 (+ 409us) server_negotiation.cc:299] Negotiation successful
0618 22:38:17.942476 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":128,"server-negotiator.queue_time_us":255}
I20260618 22:38:17.944110 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:17.930446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.18.150.189:34151)
0618 22:38:17.930584 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:17.930589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:17.930786 (+ 197us) server_negotiation.cc:407] Connection header received
0618 22:38:17.931005 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:17.931010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:17.931076 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:17.931136 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:17.932496 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.933179 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.934357 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:17.934886 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:17.935001 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:17.939276 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:17.939281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:17.939284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:17.939313 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:17.941632 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.942312 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.942318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.942321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.942452 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:17.942777 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:17.942782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:17.942785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:17.943095 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:17.943335 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:17.943654 (+ 319us) server_negotiation.cc:299] Negotiation successful
0618 22:38:17.943814 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:38:17.948287 9155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0: Bootstrap starting.
I20260618 22:38:17.948837 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:38:17.951480 9155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:17.952441 9155 log.cc:824] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:17.951378 8982 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46578
I20260618 22:38:17.954949 9155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0: No bootstrap required, opened a new log
I20260618 22:38:17.961521 9155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } }
I20260618 22:38:17.961817 9155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7529e65fe7d54a56989ec5b8b9ac4ee0, State: Initialized, Role: FOLLOWER
I20260618 22:38:17.962275 9155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } }
I20260618 22:38:17.963166 9161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } } }
I20260618 22:38:17.963582 9161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:17.964103 9155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [sys.catalog]: configured and running, proceeding with master startup.
W20260618 22:38:17.972785 9172 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:38:17.972913 9172 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:38:17.970961 9088 master_runner.cc:186] Error getting master registration for 127.18.150.190:33427: OK, instance_id { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" instance_seqno: 1781822296920030 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:38:17.978688 9088 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:38:18.054193 9007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } attempt: 4
Jun 18 22:38:18 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822296, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
I20260618 22:38:18.071537 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.055019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43128)
0618 22:38:18.055287 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:18.055310 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:18.055444 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:18.056128 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:18.056133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:18.056647 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:18.056863 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:18.056875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.058408 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.058414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:18.059871 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:18.059887 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.060859 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.060865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:18.061052 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.061718 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:18.061739 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:18.064550 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:18.068708 (+ 4158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:18.068713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:18.068718 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:18.069246 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:18.069653 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:18.069658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:18.069661 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:18.069815 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:18.070536 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:18.070541 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:18.070702 (+ 161us) client_negotiation.cc:769] Sending connection context
0618 22:38:18.070853 (+ 151us) client_negotiation.cc:240] Negotiation successful
0618 22:38:18.071132 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260618 22:38:18.071576 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.055071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43128 (local address 127.18.150.188:41373)
0618 22:38:18.055692 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.055697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.055717 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:38:18.055781 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.055786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.055860 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.055945 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.057066 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.058270 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.060023 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.060719 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.060848 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.064771 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:18.064778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:18.064783 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:18.064824 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:18.068573 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:18.069360 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:18.069366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:18.069370 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:18.069549 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:18.069923 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:18.069929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:18.069932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:18.070416 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:18.070588 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.070977 (+ 389us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.071207 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465}
I20260618 22:38:18.077435 9007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72: Bootstrap starting.
I20260618 22:38:18.080538 9007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:18.081578 9007 log.cc:824] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:18.084167 9007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72: No bootstrap required, opened a new log
I20260618 22:38:18.089504 9007 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } }
I20260618 22:38:18.089854 9007 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4972aa2a5ae40fe9c8c5eea28e5fe72, State: Initialized, Role: FOLLOWER
I20260618 22:38:18.090318 9007 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } }
I20260618 22:38:18.091185 9175 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } } }
I20260618 22:38:18.091540 9175 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:18.092273 9007 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:18.092435 8982 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260618 22:38:18.092658 8982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:18.096117 8982 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } }
I20260618 22:38:18.098765 8982 leader_election.cc:288] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [CANDIDATE]: Term 1 election: Requested vote from peers 648af6a3b9a3477fb8487a4f0171217b (127.18.150.189:34151), 7529e65fe7d54a56989ec5b8b9ac4ee0 (127.18.150.188:41373)
I20260618 22:38:18.099390 9130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0"
I20260618 22:38:18.099848 9130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:18.103236 9130 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4972aa2a5ae40fe9c8c5eea28e5fe72 in term 1.
I20260618 22:38:18.103793 8956 leader_election.cc:302] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7529e65fe7d54a56989ec5b8b9ac4ee0, c4972aa2a5ae40fe9c8c5eea28e5fe72; no voters:
I20260618 22:38:18.104228 9175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [term 1 FOLLOWER]: Leader election won for term 1
W20260618 22:38:18.114945 9109 tablet.cc:2369] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:38:18.115278 9175 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [term 1 LEADER]: Becoming Leader. State: Replica: c4972aa2a5ae40fe9c8c5eea28e5fe72, State: Running, Role: LEADER
I20260618 22:38:18.116298 9175 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } }
I20260618 22:38:18.116868 9081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } attempt: 5
Jun 18 22:38:18 dist-test-slave-h7jz krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822297, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
I20260618 22:38:18.127323 9186 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4972aa2a5ae40fe9c8c5eea28e5fe72. Latest consensus state: current_term: 1 leader_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } } }
I20260618 22:38:18.128059 9186 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:18.134810 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.117465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:41373 (local address 127.0.0.1:43132)
0618 22:38:18.117882 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:18.117902 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:18.118388 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:18.118921 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:18.118926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:18.119412 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:18.119634 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:18.119648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.120978 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.120983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:18.122438 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:18.122453 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.123580 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.123587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:18.123825 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.124524 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:18.124547 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:18.127409 (+ 2862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:18.131693 (+ 4284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:18.131699 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:18.131704 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:18.132250 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:18.132725 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:18.132731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:18.132735 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:18.132956 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:18.133693 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:18.133699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:18.133890 (+ 191us) client_negotiation.cc:769] Sending connection context
0618 22:38:18.134049 (+ 159us) client_negotiation.cc:240] Negotiation successful
0618 22:38:18.134374 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288}
I20260618 22:38:18.135872 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.118087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43132 (local address 127.18.150.188:41373)
0618 22:38:18.118246 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.118251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.118272 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:38:18.118567 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.118573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.118650 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.118736 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.119809 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.120823 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.122608 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.123254 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.123366 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.127642 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:18.127654 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:18.127660 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:18.127703 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:18.131510 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:18.132401 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:18.132407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:18.132412 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:18.132588 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:18.133090 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:18.133096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:18.133100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:18.133478 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:18.133628 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.135212 (+ 1584us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.135461 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:38:18.139173 9190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:38:18.141573 9081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b: Bootstrap starting.
I20260618 22:38:18.145570 9081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:18.147037 9081 log.cc:824] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b: Log is configured to *not* fsync() on all Append() calls
W20260618 22:38:18.160658 8961 tablet.cc:2369] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:38:18.157362 8940 master_runner.cc:186] Error getting master registration for 127.18.150.189:34151: OK, instance_id { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" instance_seqno: 1781822297391941 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:38:18.176071 9081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b: No bootstrap required, opened a new log
I20260618 22:38:18.178117 8940 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:38:18.174842 9190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:38:18.184338 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.139062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42778 (local address 127.18.150.189:34151)
0618 22:38:18.139230 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.139235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.154270 (+ 15035us) server_negotiation.cc:407] Connection header received
0618 22:38:18.154907 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.154913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.154996 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.155066 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.162061 (+ 6995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.163142 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.165097 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.165734 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.165830 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.175997 (+ 10167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:18.176003 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:18.176008 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:18.176050 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:18.179983 (+ 3933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:18.181287 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:18.181293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:18.181297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:18.181498 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:18.182199 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:18.182205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:18.182208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:18.182596 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:18.182861 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.183707 (+ 846us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.183978 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:38:18.187619 9081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } }
I20260618 22:38:18.188037 9081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648af6a3b9a3477fb8487a4f0171217b, State: Initialized, Role: FOLLOWER
I20260618 22:38:18.188572 9081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } }
I20260618 22:38:18.195165 9130 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [term 1 FOLLOWER]: Refusing update from remote peer c4972aa2a5ae40fe9c8c5eea28e5fe72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:38:18.196367 9186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:38:18.235174 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.147638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.18.150.190:33427)
0618 22:38:18.147802 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.147808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.171007 (+ 23199us) server_negotiation.cc:407] Connection header received
0618 22:38:18.171322 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.171330 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.171438 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.171553 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.182091 (+ 10538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.183187 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.186657 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.187330 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.187449 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.226173 (+ 38724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:18.226182 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:18.226187 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:18.226246 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:18.230292 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:18.231329 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:18.231341 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:18.231345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:18.231562 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:18.232162 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:18.232168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:18.232171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:18.232597 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:18.232892 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.234459 (+ 1567us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.234760 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:38:18.241425 9161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4972aa2a5ae40fe9c8c5eea28e5fe72. Latest consensus state: current_term: 1 leader_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } } }
I20260618 22:38:18.241792 9161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:18.243736 9199 mvcc.cc:205] Tried to move back new op lower bound from 7298344133395968000 to 7298344133094305792. Current Snapshot: MvccSnapshot[applied={T|T < 7298344133395968000}]
I20260618 22:38:18.250002 9201 mvcc.cc:205] Tried to move back new op lower bound from 7298344133395968000 to 7298344133094305792. Current Snapshot: MvccSnapshot[applied={T|T < 7298344133395968000}]
I20260618 22:38:18.253036 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.167606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43148 (local address 127.18.150.188:41373)
0618 22:38:18.167811 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.167816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.186092 (+ 18276us) server_negotiation.cc:407] Connection header received
0618 22:38:18.186576 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.186582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.186681 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.186747 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.189900 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.191047 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.213592 (+ 22545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.214461 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.214576 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.244743 (+ 30167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:18.244751 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:18.244757 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:18.244807 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:18.248876 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:18.250329 (+ 1453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:18.250336 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:18.250340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:18.250520 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:18.251009 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:18.251015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:18.251019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:18.251434 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:18.251699 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.252317 (+ 618us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.252637 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260618 22:38:18.255980 9161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } } }
I20260618 22:38:18.256227 9161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:18.257200 9081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:18.257670 9055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:18.255678 9200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } } }
I20260618 22:38:18.258504 9200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:18.261379 9055 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [term 1 FOLLOWER]: Refusing update from remote peer c4972aa2a5ae40fe9c8c5eea28e5fe72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:38:18.261601 9056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "648af6a3b9a3477fb8487a4f0171217b"
I20260618 22:38:18.265578 9056 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4972aa2a5ae40fe9c8c5eea28e5fe72 in term 1.
I20260618 22:38:18.266557 9186 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } } }
I20260618 22:38:18.266803 9186 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:18.268007 9175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:38:18.274148 9190 catalog_manager.cc:1347] Generated new cluster ID: f1288ee476e942989e4fe9fa72c6c580
I20260618 22:38:18.274255 9190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:38:18.295547 9190 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:38:18.297385 9190 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:38:18.322991 9190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72: Generated new TSK 0
I20260618 22:38:18.324082 9190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:38:18.359798 9200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4972aa2a5ae40fe9c8c5eea28e5fe72. Latest consensus state: current_term: 1 leader_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } } }
I20260618 22:38:18.360149 9200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:18.360419 9214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } } }
I20260618 22:38:18.360728 9214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:18.360644 9186 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4972aa2a5ae40fe9c8c5eea28e5fe72" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33427 } } peers { permanent_uuid: "648af6a3b9a3477fb8487a4f0171217b" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34151 } } peers { permanent_uuid: "7529e65fe7d54a56989ec5b8b9ac4ee0" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 41373 } } }
I20260618 22:38:18.360913 9186 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4972aa2a5ae40fe9c8c5eea28e5fe72 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:18.407389 9221 catalog_manager.cc:1259] Loaded cluster ID: f1288ee476e942989e4fe9fa72c6c580
I20260618 22:38:18.407519 9221 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b: loading cluster ID for follower catalog manager: success
I20260618 22:38:18.428705 9221 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b: acquiring CA information for follower catalog manager: success
I20260618 22:38:18.431416 9221 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 648af6a3b9a3477fb8487a4f0171217b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260618 22:38:18.449757 19034 mini_kdc.cc:348] Time spent kdestroy: real 0.113s user 0.000s sys 0.023s
W20260618 22:38:18.496505 9011 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.482823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46600 (local address 127.18.150.190:33427)
0618 22:38:18.483073 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.483080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.493871 (+ 10791us) server_negotiation.cc:407] Connection header received
0618 22:38:18.494693 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.494701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.494814 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.494911 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.496222 (+ 1311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46600: BlockingRecv error: recv got EOF from 127.0.0.1:46600 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260618 22:38:18.498894 9085 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.482866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42784 (local address 127.18.150.189:34151)
0618 22:38:18.483068 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.483075 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.497092 (+ 14017us) server_negotiation.cc:407] Connection header received
0618 22:38:18.497432 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.497440 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.497533 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.497610 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.498622 (+ 1012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42784: BlockingRecv error: recv got EOF from 127.0.0.1:42784 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260618 22:38:18.502581 9158 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.500246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43154 (local address 127.18.150.188:41373)
0618 22:38:18.500436 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.500443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.500751 (+ 308us) server_negotiation.cc:407] Connection header received
0618 22:38:18.500922 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.500929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.501013 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.501113 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.502303 (+ 1190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43154: BlockingRecv error: recv got EOF from 127.0.0.1:43154 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:38:18.527432 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.519217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42786 (local address 127.18.150.189:34151)
0618 22:38:18.519415 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.519421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.520001 (+ 580us) server_negotiation.cc:407] Connection header received
0618 22:38:18.520285 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.520296 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.520529 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.520700 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:18.521234 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.522415 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.524501 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.525211 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.525338 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.525993 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:18.526685 (+ 692us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:18.526839 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.526990 (+ 151us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.527086 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260618 22:38:18.538055 9158 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.536094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43162 (local address 127.18.150.188:41373)
0618 22:38:18.536268 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.536274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.536349 (+ 75us) server_negotiation.cc:407] Connection header received
0618 22:38:18.536715 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.536721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.536852 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.536967 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.537774 (+ 807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43162: BlockingRecv error: recv got EOF from 127.0.0.1:43162 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:38:18.550854 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.543942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.18.150.189:34151)
0618 22:38:18.544125 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.544132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.544624 (+ 492us) server_negotiation.cc:407] Connection header received
0618 22:38:18.544847 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.544854 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.544950 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.545065 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:18.545545 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.546748 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.548689 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.549401 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.549547 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.550082 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:18.550225 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:18.550314 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.550473 (+ 159us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.550552 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260618 22:38:18.564167 9158 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.558115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43176 (local address 127.18.150.188:41373)
0618 22:38:18.558321 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.558326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.562468 (+ 4142us) server_negotiation.cc:407] Connection header received
0618 22:38:18.562720 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.562728 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.562847 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.562998 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.563918 (+ 920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43176: BlockingRecv error: recv got EOF from 127.0.0.1:43176 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:38:18.580794 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.570093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42804 (local address 127.18.150.189:34151)
0618 22:38:18.570264 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.570270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.574510 (+ 4240us) server_negotiation.cc:407] Connection header received
0618 22:38:18.574699 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.574706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.574826 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.574935 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:18.575410 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.576618 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.578609 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.579322 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.579460 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.579927 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:18.580079 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:18.580186 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.580324 (+ 138us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.580418 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260618 22:38:18.594317 9158 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.591885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43178 (local address 127.18.150.188:41373)
0618 22:38:18.592085 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.592092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.592657 (+ 565us) server_negotiation.cc:407] Connection header received
0618 22:38:18.592877 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.592883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.592992 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.593103 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.594055 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43178: BlockingRecv error: recv got EOF from 127.0.0.1:43178 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260618 22:38:18.611017 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.603902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42810 (local address 127.18.150.189:34151)
0618 22:38:18.604147 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.604153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.604641 (+ 488us) server_negotiation.cc:407] Connection header received
0618 22:38:18.604836 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.604844 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.604943 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.605075 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:18.605514 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.606745 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.608687 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.609495 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.609655 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.610162 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:18.610319 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:18.610433 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.610569 (+ 136us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.610661 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
W20260618 22:38:18.628147 9158 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.623803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.18.150.188:41373)
0618 22:38:18.624052 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.624059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.626557 (+ 2498us) server_negotiation.cc:407] Connection header received
0618 22:38:18.626783 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.626789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.626885 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.627002 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.627921 (+ 919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43188: BlockingRecv error: recv got EOF from 127.0.0.1:43188 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
I20260618 22:38:18.646462 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.639545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42812 (local address 127.18.150.189:34151)
0618 22:38:18.639760 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.639767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.640339 (+ 572us) server_negotiation.cc:407] Connection header received
0618 22:38:18.640548 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.640554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.640637 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.640732 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:18.641195 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.642341 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.644297 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.644950 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.645083 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.645624 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:18.645751 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:18.645829 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.646054 (+ 225us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.646133 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260618 22:38:18.658260 9158 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.655988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.18.150.188:41373)
0618 22:38:18.656175 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.656181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.656721 (+ 540us) server_negotiation.cc:407] Connection header received
0618 22:38:18.656882 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.656888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.656992 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.657113 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.657974 (+ 861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43204: BlockingRecv error: recv got EOF from 127.0.0.1:43204 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:38:18.685340 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.677736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.18.150.189:34151)
0618 22:38:18.677935 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.677976 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.678537 (+ 561us) server_negotiation.cc:407] Connection header received
0618 22:38:18.678768 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.678777 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.678890 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.679009 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:18.679834 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.681097 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.683093 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.683872 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.684013 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.684508 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:18.684656 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:18.684768 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.684917 (+ 149us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.685032 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260618 22:38:18.694448 9158 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.692612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43216 (local address 127.18.150.188:41373)
0618 22:38:18.692869 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.692877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.692900 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:38:18.692981 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.692987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.693084 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.693210 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.694186 (+ 976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43216: BlockingRecv error: recv got EOF from 127.0.0.1:43216 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
I20260618 22:38:18.743165 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.735747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42832 (local address 127.18.150.189:34151)
0618 22:38:18.736005 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.736011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.736598 (+ 587us) server_negotiation.cc:407] Connection header received
0618 22:38:18.736880 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.736887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.736995 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.737098 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:18.737667 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.738827 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.740808 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.741578 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.741727 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.742252 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:18.742398 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:18.742495 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.742678 (+ 183us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.742783 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260618 22:38:18.751818 9158 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.749727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58888 (local address 127.18.150.188:41373)
0618 22:38:18.749992 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.749999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.750302 (+ 303us) server_negotiation.cc:407] Connection header received
0618 22:38:18.750511 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.750515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.750592 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.750681 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.751547 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58888: BlockingRecv error: recv got EOF from 127.0.0.1:58888 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:38:18.824864 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.819573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44702 (local address 127.18.150.189:34151)
0618 22:38:18.819731 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.819738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.820150 (+ 412us) server_negotiation.cc:407] Connection header received
0618 22:38:18.820336 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.820342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.820412 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.820494 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:18.820893 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.821756 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.823238 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.823694 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.823813 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.824187 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:18.824312 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:18.824387 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.824529 (+ 142us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.824611 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260618 22:38:18.833132 9158 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.831341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58892 (local address 127.18.150.188:41373)
0618 22:38:18.831551 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.831557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.831804 (+ 247us) server_negotiation.cc:407] Connection header received
0618 22:38:18.831989 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.831993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.832063 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.832142 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.832918 (+ 776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58892: BlockingRecv error: recv got EOF from 127.0.0.1:58892 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
I20260618 22:38:18.969527 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:18.964792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.18.150.189:34151)
0618 22:38:18.965044 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.965051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.965265 (+ 214us) server_negotiation.cc:407] Connection header received
0618 22:38:18.965441 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.965445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.965513 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.965609 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:18.966134 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.966806 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.968126 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:18.968515 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:18.968603 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:18.968957 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:18.969049 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:18.969106 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:18.969201 (+ 95us) server_negotiation.cc:299] Negotiation successful
0618 22:38:18.969259 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:38:18.975035 9172 catalog_manager.cc:1259] Loaded cluster ID: f1288ee476e942989e4fe9fa72c6c580
I20260618 22:38:18.975149 9172 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0: loading cluster ID for follower catalog manager: success
W20260618 22:38:18.976894 9158 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:18.974911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.18.150.188:41373)
0618 22:38:18.975118 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:18.975123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:18.975509 (+ 386us) server_negotiation.cc:407] Connection header received
0618 22:38:18.975687 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:18.975691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:18.975756 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:18.975841 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:18.976684 (+ 843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58904: BlockingRecv error: recv got EOF from 127.0.0.1:58904 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260618 22:38:18.978461 9172 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0: acquiring CA information for follower catalog manager: success
I20260618 22:38:18.980163 9172 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7529e65fe7d54a56989ec5b8b9ac4ee0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:38:19.244141 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:19.239326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.18.150.189:34151)
0618 22:38:19.239485 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:19.239489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:19.239738 (+ 249us) server_negotiation.cc:407] Connection header received
0618 22:38:19.239929 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:19.239934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:19.240018 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:19.240102 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:19.240436 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:19.241144 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:19.242653 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:19.243056 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:19.243142 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:19.243552 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:19.243657 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:19.243712 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:19.243867 (+ 155us) server_negotiation.cc:299] Negotiation successful
0618 22:38:19.243943 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:38:19.255013 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:19.249435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58918 (local address 127.18.150.188:41373)
0618 22:38:19.249615 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:19.249621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:19.249880 (+ 259us) server_negotiation.cc:407] Connection header received
0618 22:38:19.250113 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:19.250118 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:19.250277 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:19.250375 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:19.250755 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:19.251487 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:19.252961 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:19.253340 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:19.253443 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:19.253907 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:19.254457 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:19.254589 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:19.254676 (+ 87us) server_negotiation.cc:299] Negotiation successful
0618 22:38:19.254761 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:38:19.268451 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:19.260744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.18.150.189:34151)
0618 22:38:19.260949 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:19.260956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:19.262428 (+ 1472us) server_negotiation.cc:407] Connection header received
0618 22:38:19.262658 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:19.262664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:19.262756 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:19.262859 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:19.263679 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:19.264748 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:19.266675 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:19.267075 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:19.267185 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:19.267733 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:19.267859 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:19.267952 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:19.268069 (+ 117us) server_negotiation.cc:299] Negotiation successful
0618 22:38:19.268148 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:38:19.269568 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:19.261382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58922 (local address 127.18.150.188:41373)
0618 22:38:19.261582 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:19.261587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:19.262672 (+ 1085us) server_negotiation.cc:407] Connection header received
0618 22:38:19.262893 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:19.262899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:19.262974 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:19.263021 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:19.263839 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:19.264909 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:19.267546 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:19.268108 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:19.268190 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:19.268877 (+ 687us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:19.269004 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:19.269090 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:19.269216 (+ 126us) server_negotiation.cc:299] Negotiation successful
0618 22:38:19.269289 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260618 22:38:19.270311 9335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:19.260659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33546 (local address 127.18.150.190:33427)
0618 22:38:19.262973 (+ 2314us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:19.262979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:19.262996 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:38:19.263102 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:19.263108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:19.263337 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:19.263526 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:19.264084 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:19.264905 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:19.267553 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:19.268022 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:19.268478 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:19.269104 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:19.269713 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:19.269815 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:19.269916 (+ 101us) server_negotiation.cc:299] Negotiation successful
0618 22:38:19.270020 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2154,"thread_start_us":2065,"threads_started":1}
I20260618 22:38:19.274315 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 8940
I20260618 22:38:19.275693 9002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:38:19.382498 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 8940
I20260618 22:38:19.399066 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 9014
I20260618 22:38:19.400070 9076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:38:19.506484 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 9014
I20260618 22:38:19.526659 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 9088
I20260618 22:38:19.527642 9150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:38:19.631762 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 9088
2026-06-18T22:38:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4369 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:38:19 dist-test-slave-h7jz krb5kdc[9343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:38:19 dist-test-slave-h7jz krb5kdc[9343](info): set up 2 sockets
Jun 18 22:38:19 dist-test-slave-h7jz krb5kdc[9343](info): commencing operation
krb5kdc: starting...
W20260618 22:38:20.654455 19034 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/test-user.keytab.
Jun 18 22:38:20 dist-test-slave-h7jz krb5kdc[9343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:38:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:38:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.150.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
Entry for principal kudu/127.18.150.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab.
I20260618 22:38:20.892632 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33827
--webserver_interface=127.18.150.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:33827,127.18.150.189:34143,127.18.150.188:36067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:38:21.207770 9356 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:38:21.208101 9356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:21.208195 9356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:21.208308 9356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:21.215454 9356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:21.215548 9356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:21.215637 9356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:21.215699 9356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:21.215780 9356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:21.225315 9356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:33827,127.18.150.189:34143,127.18.150.188:36067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.190.keytab
--principal=kudu/127.18.150.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.190:33827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.150.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:21.226604 9356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:21.227869 9356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:21.239144 9361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:21.239179 9364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:21.239212 9362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:21.239990 9356 server_base.cc:1034] running on GCE node
I20260618 22:38:21.240778 9356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:21.242193 9356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:21.243433 9356 hybrid_clock.cc:648] HybridClock initialized: now 1781822301243356 us; error 49 us; skew 500 ppm
I20260618 22:38:21.243746 9356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:21 dist-test-slave-h7jz krb5kdc[9343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822301, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:21.249548 9356 init.cc:377] Logged in from keytab as kudu/127.18.150.190@KRBTEST.COM (short username kudu)
I20260618 22:38:21.251109 9356 webserver.cc:460] Webserver started at http://127.18.150.190:46105/ using document root <none> and password file <none>
I20260618 22:38:21.251860 9356 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:21.251969 9356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:21.252300 9356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:21.255514 9356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/data/instance:
uuid: "6c8f0ccf6b584d88af43e96e56374695"
format_stamp: "Formatted at 2026-06-18 22:38:21 on dist-test-slave-h7jz"
server_key: "1f9f0af6a8ac70f3c42775337906f875"
server_key_iv: "c5b18fdb89ec30b5eecde58c4fe7adeb"
server_key_version: "encryptionkey@0"
I20260618 22:38:21.256336 9356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/wal/instance:
uuid: "6c8f0ccf6b584d88af43e96e56374695"
format_stamp: "Formatted at 2026-06-18 22:38:21 on dist-test-slave-h7jz"
server_key: "1f9f0af6a8ac70f3c42775337906f875"
server_key_iv: "c5b18fdb89ec30b5eecde58c4fe7adeb"
server_key_version: "encryptionkey@0"
I20260618 22:38:21.261606 9356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260618 22:38:21.264991 9371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:21.266387 9356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:38:21.266538 9356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/wal
uuid: "6c8f0ccf6b584d88af43e96e56374695"
format_stamp: "Formatted at 2026-06-18 22:38:21 on dist-test-slave-h7jz"
server_key: "1f9f0af6a8ac70f3c42775337906f875"
server_key_iv: "c5b18fdb89ec30b5eecde58c4fe7adeb"
server_key_version: "encryptionkey@0"
I20260618 22:38:21.266723 9356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:21.294608 9356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:21.298844 9356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:21.299088 9356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:21.315981 9356 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.190:33827
I20260618 22:38:21.316000 9422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.190:33827 every 8 connection(s)
I20260618 22:38:21.317602 9356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/data/info.pb
I20260618 22:38:21.321391 9423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:21.326078 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 9356
I20260618 22:38:21.326239 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-0/wal/instance
I20260618 22:38:21.326630 19034 external_mini_cluster.cc:1428] Setting key 35b520dc82865ad9ee0d5f19532cd25f
I20260618 22:38:21.327813 9423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:38:21 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822301, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.150.189@KRBTEST.COM; defaulting to no policy
I20260618 22:38:21.349498 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.331630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53368 (local address 127.18.150.190:33827)
0618 22:38:21.332192 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:21.332198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:21.332236 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:38:21.334072 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:21.334078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:21.334311 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:21.334462 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:21.335630 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.337319 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.338681 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.339413 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.339579 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:21.342749 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:21.342757 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:21.342768 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:21.342799 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:21.345380 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:21.346236 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:21.346263 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:21.346265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:21.346422 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:21.346833 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:21.346836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:21.346838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:21.347332 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:21.347483 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:21.348108 (+ 625us) server_negotiation.cc:299] Negotiation successful
0618 22:38:21.348406 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":406,"thread_start_us":273,"threads_started":1}
I20260618 22:38:21.349532 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.330641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33827 (local address 127.0.0.1:53368)
0618 22:38:21.331389 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.331435 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:21.333672 (+ 2237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:21.334528 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:21.334535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:21.335099 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:21.335432 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:21.335448 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.337456 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.337459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:21.338557 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:21.338568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.339596 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.339601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:21.339737 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:21.340303 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:21.340333 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:21.342559 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:21.345500 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:21.345512 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:21.345524 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:21.346071 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:21.346536 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:21.346541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:21.346545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:21.346740 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:21.347468 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:21.347476 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:21.347711 (+ 235us) client_negotiation.cc:769] Sending connection context
0618 22:38:21.347986 (+ 275us) client_negotiation.cc:240] Negotiation successful
0618 22:38:21.348565 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":111,"thread_start_us":274,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.189@KRBTEST.COM" created.
I20260618 22:38:21.352515 9423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:38:21.353632 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.353248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58818)
0618 22:38:21.353404 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.353534 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260618 22:38:21.353996 9374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
I20260618 22:38:21.355468 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.355161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58820)
0618 22:38:21.355283 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.355346 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260618 22:38:21.355651 9423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:34143: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab.
Entry for principal kudu/127.18.150.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab.
I20260618 22:38:21.379212 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.189:34143
--webserver_interface=127.18.150.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab
--principal=kudu/127.18.150.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:33827,127.18.150.189:34143,127.18.150.188:36067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:38:21.404948 9423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } attempt: 1
I20260618 22:38:21.406026 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.405566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58836)
0618 22:38:21.405750 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.405866 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260618 22:38:21.407025 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.406691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58850)
0618 22:38:21.406832 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.406897 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260618 22:38:21.407212 9423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:34143: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
I20260618 22:38:21.475466 9423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } attempt: 2
I20260618 22:38:21.476662 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.476170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58856)
0618 22:38:21.476408 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.476526 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260618 22:38:21.477620 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.477264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58866)
0618 22:38:21.477412 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.477490 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260618 22:38:21.477798 9423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:34143: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
I20260618 22:38:21.569025 9423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } attempt: 3
I20260618 22:38:21.570144 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.569670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58870)
0618 22:38:21.569863 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.569991 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260618 22:38:21.571151 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.570785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58882)
0618 22:38:21.570930 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.571001 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260618 22:38:21.571372 9423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:34143: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
W20260618 22:38:21.694267 9430 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:38:21.694617 9430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:21.694712 9430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:21.694816 9430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:21.702257 9430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:21.702356 9430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:21.702440 9430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:21.702502 9430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:21.702589 9430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:21.712430 9430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:33827,127.18.150.189:34143,127.18.150.188:36067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.189.keytab
--principal=kudu/127.18.150.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.189:34143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.150.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:21.713656 9430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:21.714615 9423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } attempt: 4
I20260618 22:38:21.715042 9430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:38:21.715731 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.715216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58896)
0618 22:38:21.715518 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.715604 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260618 22:38:21.716691 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.716362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58904)
0618 22:38:21.716500 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.716577 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260618 22:38:21.716878 9423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.189:34143: Network error: Client connection negotiation failed: client connection to 127.18.150.189:34143: connect: Connection refused (error 111)
W20260618 22:38:21.726219 9435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:21.726632 9436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:21.726678 9438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:21.727458 9430 server_base.cc:1034] running on GCE node
I20260618 22:38:21.728169 9430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:21.729856 9430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:21.731125 9430 hybrid_clock.cc:648] HybridClock initialized: now 1781822301731002 us; error 90 us; skew 500 ppm
I20260618 22:38:21.731521 9430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:21 dist-test-slave-h7jz krb5kdc[9343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822301, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:21.734573 9430 init.cc:377] Logged in from keytab as kudu/127.18.150.189@KRBTEST.COM (short username kudu)
I20260618 22:38:21.735982 9430 webserver.cc:460] Webserver started at http://127.18.150.189:38705/ using document root <none> and password file <none>
I20260618 22:38:21.736661 9430 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:21.736754 9430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:21.737058 9430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:21.739993 9430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/data/instance:
uuid: "2776e2171478463ba416e25491e1c119"
format_stamp: "Formatted at 2026-06-18 22:38:21 on dist-test-slave-h7jz"
server_key: "19476da103b90a6ba47c899a35c3b006"
server_key_iv: "89ed3746b09ac3f794f2d17be2ffec2c"
server_key_version: "encryptionkey@0"
I20260618 22:38:21.740782 9430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/wal/instance:
uuid: "2776e2171478463ba416e25491e1c119"
format_stamp: "Formatted at 2026-06-18 22:38:21 on dist-test-slave-h7jz"
server_key: "19476da103b90a6ba47c899a35c3b006"
server_key_iv: "89ed3746b09ac3f794f2d17be2ffec2c"
server_key_version: "encryptionkey@0"
I20260618 22:38:21.746004 9430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260618 22:38:21.749325 9445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:21.750728 9430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:38:21.750882 9430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/wal
uuid: "2776e2171478463ba416e25491e1c119"
format_stamp: "Formatted at 2026-06-18 22:38:21 on dist-test-slave-h7jz"
server_key: "19476da103b90a6ba47c899a35c3b006"
server_key_iv: "89ed3746b09ac3f794f2d17be2ffec2c"
server_key_version: "encryptionkey@0"
I20260618 22:38:21.751071 9430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:21.767166 9430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:21.771198 9430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:21.771426 9430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:21.787771 9430 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.189:34143
I20260618 22:38:21.787788 9496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.189:34143 every 8 connection(s)
I20260618 22:38:21.789036 9430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/data/info.pb
I20260618 22:38:21.791828 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 9430
I20260618 22:38:21.792007 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-1/wal/instance
I20260618 22:38:21.792574 19034 external_mini_cluster.cc:1428] Setting key 336d478b299320418e56a3b01fe99a2c
I20260618 22:38:21.794301 9497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:21.800788 9497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.150.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.150.188@KRBTEST.COM" created.
Jun 18 22:38:21 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822301, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:38:21.822921 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.803437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53382 (local address 127.18.150.190:33827)
0618 22:38:21.803641 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:21.803648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:21.804871 (+ 1223us) server_negotiation.cc:407] Connection header received
0618 22:38:21.806878 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:21.806885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:21.806981 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:21.807096 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:21.808717 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.809690 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.811315 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.811970 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.812107 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:21.817013 (+ 4906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:21.817021 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:21.817026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:21.817071 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:21.819999 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:21.820754 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:21.820760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:21.820763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:21.820913 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:21.821254 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:21.821257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:21.821260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:21.821579 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:21.821720 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:21.822400 (+ 680us) server_negotiation.cc:299] Negotiation successful
0618 22:38:21.822616 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260618 22:38:21.823776 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.803904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33827 (local address 127.0.0.1:53382)
0618 22:38:21.804687 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.804744 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:21.806593 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:21.807378 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:21.807387 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:21.807940 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:21.808532 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:21.808550 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.809846 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.809850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:21.811154 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:21.811167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.812180 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.812185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:21.812421 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:21.813064 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:21.813087 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:21.816732 (+ 3645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:21.820153 (+ 3421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:21.820161 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:21.820175 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:21.820634 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:21.820999 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:21.821002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:21.821004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:21.821165 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:21.821676 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:21.821683 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:21.822057 (+ 374us) client_negotiation.cc:769] Sending connection context
0618 22:38:21.822284 (+ 227us) client_negotiation.cc:240] Negotiation successful
0618 22:38:21.822675 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":325,"threads_started":1}
I20260618 22:38:21.825664 9497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:38:21 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822301, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
I20260618 22:38:21.840631 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.826374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58916)
0618 22:38:21.826558 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.826575 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:21.826670 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:21.827698 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:21.827702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:21.828115 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:21.828282 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:21.828293 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.829681 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.829686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:21.830887 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:21.830901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.831683 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.831688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:21.831853 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:21.832403 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:21.832422 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:21.834726 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:21.838065 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:21.838070 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:21.838075 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:21.838574 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:21.838963 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:21.838967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:21.838971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:21.839144 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:21.839703 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:21.839708 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:21.839861 (+ 153us) client_negotiation.cc:769] Sending connection context
0618 22:38:21.840009 (+ 148us) client_negotiation.cc:240] Negotiation successful
0618 22:38:21.840299 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260618 22:38:21.840710 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.826641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.18.150.189:34143)
0618 22:38:21.827287 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:21.827303 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:21.827327 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:38:21.827371 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:21.827377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:21.827552 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:21.827667 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:21.828464 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.829548 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.831011 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:21.831549 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:21.831641 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:21.834930 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:21.834943 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:21.834958 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:21.834986 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:21.837905 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:21.838712 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:21.838719 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:21.838722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:21.838860 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:21.839237 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:21.839241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:21.839243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:21.839575 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:21.839732 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:21.840126 (+ 394us) server_negotiation.cc:299] Negotiation successful
0618 22:38:21.840425 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":496,"thread_start_us":369,"threads_started":1}
I20260618 22:38:21.842985 9497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:38:21.843954 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.843551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52638)
0618 22:38:21.843705 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.843817 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260618 22:38:21.844250 9449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
I20260618 22:38:21.845678 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.845371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52652)
0618 22:38:21.845513 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.845584 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260618 22:38:21.845898 9497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:36067: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.150.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab.
Entry for principal kudu/127.18.150.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab.
I20260618 22:38:21.847483 19034 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8lyoIK/build/asan/bin/kudu
/tmp/dist-test-task8lyoIK/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.188:36067
--webserver_interface=127.18.150.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab
--principal=kudu/127.18.150.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.150.148:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.150.190:33827,127.18.150.189:34143,127.18.150.188:36067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:38:21.895267 9497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } attempt: 1
I20260618 22:38:21.896589 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.896013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52656)
0618 22:38:21.896249 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.896391 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260618 22:38:21.897641 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.897250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52666)
0618 22:38:21.897413 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.897512 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260618 22:38:21.897850 9497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:36067: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
I20260618 22:38:21.966094 9497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } attempt: 2
I20260618 22:38:21.967329 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.966799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52670)
0618 22:38:21.967021 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.967137 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260618 22:38:21.968416 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:21.967995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52674)
0618 22:38:21.968178 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:21.968275 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260618 22:38:21.968631 9497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:36067: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
I20260618 22:38:22.016120 9423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } attempt: 5
Jun 18 22:38:22 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822301, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
I20260618 22:38:22.030759 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.017171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58922 (local address 127.18.150.189:34143)
0618 22:38:22.017343 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.017348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.017394 (+ 46us) server_negotiation.cc:407] Connection header received
0618 22:38:22.017450 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.017454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.017515 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.017595 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.018622 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.019370 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.021087 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.021562 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.021659 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.025433 (+ 3774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.025439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.025442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.025476 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.028015 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.028764 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.028768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.028770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.028894 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.029296 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.029301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.029304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.029675 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.029838 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.030371 (+ 533us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.030548 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260618 22:38:22.030886 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.016896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58922)
0618 22:38:22.017133 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.017154 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:22.017292 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:22.017659 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:22.017664 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:22.018206 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:22.018442 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.018455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.019519 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.019525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.020909 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.020927 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.021715 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.021722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.021971 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.022658 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:22.022684 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:22.025236 (+ 2552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:22.028150 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.028155 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.028159 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.028642 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.028992 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.028998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.029001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.029168 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.029831 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:22.029837 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:22.030070 (+ 233us) client_negotiation.cc:769] Sending connection context
0618 22:38:22.030231 (+ 161us) client_negotiation.cc:240] Negotiation successful
0618 22:38:22.030538 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260618 22:38:22.031919 9423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:38:22.033073 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.032605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52676)
0618 22:38:22.032800 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.032910 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260618 22:38:22.034199 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.033772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52684)
0618 22:38:22.033968 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.034059 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260618 22:38:22.034423 9423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:36067: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
I20260618 22:38:22.059863 9497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } attempt: 3
I20260618 22:38:22.061070 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.060549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52694)
0618 22:38:22.060757 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.060886 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260618 22:38:22.062156 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.061672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52708)
0618 22:38:22.061899 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.062022 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260618 22:38:22.062404 9497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:36067: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
I20260618 22:38:22.085662 9423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } attempt: 1
I20260618 22:38:22.086876 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.086330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52712)
0618 22:38:22.086554 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.086662 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260618 22:38:22.087889 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.087485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52726)
0618 22:38:22.087657 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.087746 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260618 22:38:22.088102 9423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:36067: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
I20260618 22:38:22.156308 9423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } attempt: 2
I20260618 22:38:22.157444 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.156894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52734)
0618 22:38:22.157136 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.157245 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260618 22:38:22.158492 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.158082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52744)
0618 22:38:22.158227 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.158347 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260618 22:38:22.158718 9423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:36067: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
W20260618 22:38:22.167003 9504 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:38:22.167297 9504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:38:22.167393 9504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:38:22.167476 9504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:38:22.174860 9504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:38:22.174950 9504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:38:22.175010 9504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:38:22.175060 9504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:38:22.175117 9504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:38:22.184859 9504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.150.148:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.150.190:33827,127.18.150.189:34143,127.18.150.188:36067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/krb5kdc/kudu_127.18.150.188.keytab
--principal=kudu/127.18.150.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.150.188:36067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.150.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 18 Jun 2026 21:43:06 UTC on ccycloud.quasar-uighkb.root.comops.site
build id 4454736
ASAN enabled
I20260618 22:38:22.186156 9504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:38:22.187412 9504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:38:22.198236 9512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:22.198277 9510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:38:22.198299 9509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:38:22.198966 9504 server_base.cc:1034] running on GCE node
I20260618 22:38:22.199687 9504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:38:22.200929 9504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:38:22.202131 9504 hybrid_clock.cc:648] HybridClock initialized: now 1781822302202069 us; error 34 us; skew 500 ppm
I20260618 22:38:22.202431 9504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 18 22:38:22 dist-test-slave-h7jz krb5kdc[9343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:38:22.205267 9504 init.cc:377] Logged in from keytab as kudu/127.18.150.188@KRBTEST.COM (short username kudu)
I20260618 22:38:22.205613 9497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } attempt: 4
I20260618 22:38:22.206775 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.206232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52760)
0618 22:38:22.206470 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.206571 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260618 22:38:22.206847 9504 webserver.cc:460] Webserver started at http://127.18.150.188:46313/ using document root <none> and password file <none>
I20260618 22:38:22.207650 9504 fs_manager.cc:359] Metadata directory not provided
I20260618 22:38:22.207767 9504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:38:22.207780 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.207421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52768)
0618 22:38:22.207567 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.207657 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260618 22:38:22.208005 9497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.150.188:36067: Network error: Client connection negotiation failed: client connection to 127.18.150.188:36067: connect: Connection refused (error 111)
I20260618 22:38:22.208112 9504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:38:22.210883 9504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/data/instance:
uuid: "35cb7cba29f741a3a87990ea8d399fff"
format_stamp: "Formatted at 2026-06-18 22:38:22 on dist-test-slave-h7jz"
server_key: "0cb5fa63a53f90d2a4e78400e227e8ab"
server_key_iv: "41b96cd34eea82b2c237cf8506e11e73"
server_key_version: "encryptionkey@0"
I20260618 22:38:22.211591 9504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/wal/instance:
uuid: "35cb7cba29f741a3a87990ea8d399fff"
format_stamp: "Formatted at 2026-06-18 22:38:22 on dist-test-slave-h7jz"
server_key: "0cb5fa63a53f90d2a4e78400e227e8ab"
server_key_iv: "41b96cd34eea82b2c237cf8506e11e73"
server_key_version: "encryptionkey@0"
I20260618 22:38:22.216225 9504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260618 22:38:22.219698 9519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:38:22.220978 9504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:38:22.221122 9504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/data,/tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/wal
uuid: "35cb7cba29f741a3a87990ea8d399fff"
format_stamp: "Formatted at 2026-06-18 22:38:22 on dist-test-slave-h7jz"
server_key: "0cb5fa63a53f90d2a4e78400e227e8ab"
server_key_iv: "41b96cd34eea82b2c237cf8506e11e73"
server_key_version: "encryptionkey@0"
I20260618 22:38:22.221309 9504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:38:22.242029 9504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:38:22.246452 9504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:38:22.246701 9504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:38:22.263389 9504 rpc_server.cc:307] RPC server started. Bound to: 127.18.150.188:36067
I20260618 22:38:22.263420 9570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.150.188:36067 every 8 connection(s)
I20260618 22:38:22.264644 9504 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/data/info.pb
I20260618 22:38:22.264925 9423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } attempt: 3
I20260618 22:38:22.268780 9571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:38:22.270241 19034 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8lyoIK/build/asan/bin/kudu as pid 9504
I20260618 22:38:22.270433 19034 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8lyoIK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821960891835-19034-0/minicluster-data/master-2/wal/instance
I20260618 22:38:22.270938 19034 external_mini_cluster.cc:1428] Setting key 269fd0498f15baf88ecdae2ac80dc281
I20260618 22:38:22.278093 9571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:38:22 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822301, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.190@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
I20260618 22:38:22.288600 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.265615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52774)
0618 22:38:22.265870 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.265895 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:22.266028 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:22.270294 (+ 4266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:22.270299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:22.270793 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:22.271043 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.271058 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.273479 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.273485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.275039 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.275058 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.276522 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.276529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.276736 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.277444 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:22.277470 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:22.280371 (+ 2901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:22.285441 (+ 5070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.285447 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.285451 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.285975 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.286466 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.286470 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.286473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.286644 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.287609 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:22.287615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:22.287792 (+ 177us) client_negotiation.cc:769] Sending connection context
0618 22:38:22.287941 (+ 149us) client_negotiation.cc:240] Negotiation successful
0618 22:38:22.288193 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260618 22:38:22.289892 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.267106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.18.150.188:36067)
0618 22:38:22.267829 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.267840 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.267888 (+ 48us) server_negotiation.cc:407] Connection header received
0618 22:38:22.269509 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.269531 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.269963 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.270318 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.271208 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.273278 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.275253 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.276338 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.280609 (+ 4271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.281003 (+ 394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.281014 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.281037 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.281076 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.285291 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.286122 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.286130 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.286138 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.286336 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.286790 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.286795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.286798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.287470 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.287647 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.288219 (+ 572us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.288640 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":322,"threads_started":1}
Jun 18 22:38:22 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
Jun 18 22:38:22 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
I20260618 22:38:22.297503 9423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695: Bootstrap starting.
I20260618 22:38:22.298002 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.281121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.190:33827 (local address 127.0.0.1:53390)
0618 22:38:22.283357 (+ 2236us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.283422 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:22.283814 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:22.284302 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:22.284309 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:22.285232 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:22.285590 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.285598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.286784 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.286787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.288164 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.288177 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.289292 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.289296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.289426 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.289958 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:22.289982 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:22.292287 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:22.295557 (+ 3270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.295568 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.295574 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.296032 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.296377 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.296382 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.296386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.296514 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.296995 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:22.297001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:22.297154 (+ 153us) client_negotiation.cc:769] Sending connection context
0618 22:38:22.297301 (+ 147us) client_negotiation.cc:240] Negotiation successful
0618 22:38:22.297586 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2002,"thread_start_us":1853,"threads_started":1}
I20260618 22:38:22.298444 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.281680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.18.150.190:33827)
0618 22:38:22.281858 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.281875 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.283503 (+ 1628us) server_negotiation.cc:407] Connection header received
0618 22:38:22.283966 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.283972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.284044 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.284108 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.285716 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.286631 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.288306 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.288968 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.289048 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.292471 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.292477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.292481 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.292519 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.295443 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.296134 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.296140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.296143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.296271 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.296608 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.296613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.296616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.296880 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.297004 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.297421 (+ 417us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.297673 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:38:22.299566 9571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:38:22.299839 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.271712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52786 (local address 127.18.150.188:36067)
0618 22:38:22.282854 (+ 11142us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.282859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.283486 (+ 627us) server_negotiation.cc:407] Connection header received
0618 22:38:22.283734 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.283739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.283794 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.283888 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.285199 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.286356 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.288080 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.288675 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.288760 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.293146 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.293151 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.293155 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.293189 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.296272 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.297050 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.297055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.297058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.297200 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.297622 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.297627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.297630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.297959 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.298235 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.299225 (+ 990us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.299433 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10969,"thread_start_us":11158,"threads_started":1}
I20260618 22:38:22.301401 9423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:22.302747 9423 log.cc:824] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:22.306645 9423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695: No bootstrap required, opened a new log
Jun 18 22:38:22 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
Jun 18 22:38:22 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.190@KRBTEST.COM
I20260618 22:38:22.313683 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.300157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.189:34143 (local address 127.0.0.1:58932)
0618 22:38:22.300456 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.300476 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:22.300581 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:22.300909 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:22.300913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:22.301322 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:22.301498 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.301510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.302893 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.302898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.303904 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.303915 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.304582 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.304588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.304768 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.305333 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:22.305353 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:22.308046 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:22.311270 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.311273 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.311276 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.311722 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.312127 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.312132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.312136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.312286 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.312796 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:22.312801 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:22.312964 (+ 163us) client_negotiation.cc:769] Sending connection context
0618 22:38:22.313116 (+ 152us) client_negotiation.cc:240] Negotiation successful
0618 22:38:22.313362 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260618 22:38:22.313810 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.300213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58932 (local address 127.18.150.189:34143)
0618 22:38:22.300370 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.300375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.300550 (+ 175us) server_negotiation.cc:407] Connection header received
0618 22:38:22.300683 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.300689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.300770 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.300881 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.301667 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.302697 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.304037 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.304451 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.304558 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.308246 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.308253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.308257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.308295 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.311133 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.311844 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.311850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.311853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.312016 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.312417 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.312422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.312426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.312688 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.312818 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.313236 (+ 418us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.313459 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:38:22.314741 9571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:38:22.316110 9423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } }
I20260618 22:38:22.316490 9423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8f0ccf6b584d88af43e96e56374695, State: Initialized, Role: FOLLOWER
I20260618 22:38:22.317005 9423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } }
I20260618 22:38:22.317798 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.301598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53406 (local address 127.18.150.190:33827)
0618 22:38:22.301796 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.301800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.302001 (+ 201us) server_negotiation.cc:407] Connection header received
0618 22:38:22.302190 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.302195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.302254 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.302327 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.303447 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.304378 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.305664 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.306191 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.306296 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.310553 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.310559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.310564 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.310602 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.314758 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.315517 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.315525 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.315529 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.315696 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.316135 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.316140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.316143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.316475 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.316703 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.317180 (+ 477us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.317509 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
Jun 18 22:38:22 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.150.189@KRBTEST.COM
Jun 18 22:38:22 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.188@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
I20260618 22:38:22.333279 9578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } } }
I20260618 22:38:22.333693 9578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:22.334616 9423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:22.335196 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.319253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58942 (local address 127.18.150.189:34143)
0618 22:38:22.319408 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.319414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.319433 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:38:22.319532 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.319537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.319598 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.319671 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.320742 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.321449 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.322879 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.323485 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.323598 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.329038 (+ 5440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.329045 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.329050 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.329087 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.332417 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.333196 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.333201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.333204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.333352 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.333775 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.333780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.333783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.334153 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.334404 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.334660 (+ 256us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.334873 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:38:22.335933 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.315540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52798)
0618 22:38:22.316136 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.316153 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:22.316238 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:22.322424 (+ 6186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:22.322430 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:22.322871 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:22.323055 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.323065 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.324281 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.324287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.325500 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.325514 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.326580 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.326587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.326792 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.327318 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:22.327340 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:22.329839 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:22.333186 (+ 3347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.333190 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.333193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.333744 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.334186 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.334191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.334194 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.334340 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.335015 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:22.335018 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:22.335158 (+ 140us) client_negotiation.cc:769] Sending connection context
0618 22:38:22.335313 (+ 155us) client_negotiation.cc:240] Negotiation successful
0618 22:38:22.335598 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500}
I20260618 22:38:22.336803 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.315986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52798 (local address 127.18.150.188:36067)
0618 22:38:22.322117 (+ 6131us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.322123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.322142 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:38:22.322191 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.322196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.322276 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.322390 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.323216 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.324147 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.325634 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.326277 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.326369 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.330038 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.330043 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.330048 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.330081 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.333059 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.333908 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.333913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.333916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.334068 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.334474 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.334478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.334481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.334839 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.334960 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.336213 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.336451 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5995}
I20260618 22:38:22.340231 19034 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:38:22.342449 9398 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6c8f0ccf6b584d88af43e96e56374695"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53406
I20260618 22:38:22.342716 9571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff: Bootstrap starting.
I20260618 22:38:22.342841 9398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260618 22:38:22.343019 9398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:22.346410 9571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:22.346664 9398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } }
W20260618 22:38:22.346958 9589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:38:22.347065 9589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:38:22.347668 9571 log.cc:824] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:22.349529 9398 leader_election.cc:288] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [CANDIDATE]: Term 1 election: Requested vote from peers 2776e2171478463ba416e25491e1c119 (127.18.150.189:34143), 35cb7cba29f741a3a87990ea8d399fff (127.18.150.188:36067)
I20260618 22:38:22.351511 9356 master_runner.cc:186] Error getting master registration for 127.18.150.189:34143: OK, instance_id { permanent_uuid: "2776e2171478463ba416e25491e1c119" instance_seqno: 1781822301783066 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:38:22.366487 9356 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:38:22.370678 9571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff: No bootstrap required, opened a new log
I20260618 22:38:22.386667 9571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } }
I20260618 22:38:22.387427 9571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35cb7cba29f741a3a87990ea8d399fff, State: Initialized, Role: FOLLOWER
I20260618 22:38:22.388314 9571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } }
I20260618 22:38:22.422552 9599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } } }
I20260618 22:38:22.430699 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.376400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53420 (local address 127.18.150.190:33827)
0618 22:38:22.376565 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.376570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.389418 (+ 12848us) server_negotiation.cc:407] Connection header received
0618 22:38:22.389870 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.389876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.389985 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.390064 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.398952 (+ 8888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.400114 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.404852 (+ 4738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.405588 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.405705 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.412083 (+ 6378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.412090 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.412095 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.412151 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.418246 (+ 6095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.427634 (+ 9388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.427640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.427643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.427858 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.428364 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.428370 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.428374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.428757 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.429043 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.430047 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.430317 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260618 22:38:22.433598 9599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:22.423993 9546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6c8f0ccf6b584d88af43e96e56374695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "35cb7cba29f741a3a87990ea8d399fff"
I20260618 22:38:22.434096 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.375838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.18.150.188:36067)
0618 22:38:22.376256 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.376263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.390405 (+ 14142us) server_negotiation.cc:407] Connection header received
0618 22:38:22.390633 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.390639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.390718 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.390787 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.401782 (+ 10995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.402892 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.407665 (+ 4773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.408393 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.408516 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.425554 (+ 17038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.425560 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.425564 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.425609 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.429060 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.430382 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.430390 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.430394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.430639 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.431516 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.431522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.431525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.432127 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.432470 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.433127 (+ 657us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.433503 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285}
I20260618 22:38:22.423955 9571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:22.436131 9546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:22.441387 9546 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8f0ccf6b584d88af43e96e56374695 in term 1.
I20260618 22:38:22.442137 9375 leader_election.cc:302] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35cb7cba29f741a3a87990ea8d399fff, 6c8f0ccf6b584d88af43e96e56374695; no voters:
I20260618 22:38:22.442713 9578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:38:22.444248 9502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.380504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58948 (local address 127.18.150.189:34143)
0618 22:38:22.380672 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.380678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.423570 (+ 42892us) server_negotiation.cc:407] Connection header received
0618 22:38:22.425672 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.425677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.425759 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.425830 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.428482 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.429603 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.431832 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.432506 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.432602 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.436417 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.436425 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.436430 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.436472 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.439868 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.440665 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.440671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.440674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.440824 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.441215 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.441220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.441224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.441684 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.441908 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.442567 (+ 659us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.443891 (+ 1324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:38:22.455068 9578 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [term 1 LEADER]: Becoming Leader. State: Replica: 6c8f0ccf6b584d88af43e96e56374695, State: Running, Role: LEADER
I20260618 22:38:22.457226 9578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } }
I20260618 22:38:22.461593 9601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c8f0ccf6b584d88af43e96e56374695. Latest consensus state: current_term: 1 leader_uuid: "6c8f0ccf6b584d88af43e96e56374695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } } }
I20260618 22:38:22.462198 9601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:22.468457 9604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:38:22.471318 9604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:38:22.477360 9504 master_runner.cc:186] Error getting master registration for 127.18.150.189:34143: OK, instance_id { permanent_uuid: "2776e2171478463ba416e25491e1c119" instance_seqno: 1781822301783066 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:38:22.484825 9504 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260618 22:38:22.492103 9613 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:38:22.492221 9613 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:38:22.492349 9546 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [term 1 FOLLOWER]: Refusing update from remote peer 6c8f0ccf6b584d88af43e96e56374695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:38:22.494793 9578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:38:22.498805 9525 tablet.cc:2369] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:38:22.507208 9497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } attempt: 5
I20260618 22:38:22.518338 9599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c8f0ccf6b584d88af43e96e56374695. Latest consensus state: current_term: 1 leader_uuid: "6c8f0ccf6b584d88af43e96e56374695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } } }
I20260618 22:38:22.518621 9599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [sys.catalog]: This master's current role is: FOLLOWER
Jun 18 22:38:22 dist-test-slave-h7jz krb5kdc[9343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781822301, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.150.189@KRBTEST.COM for kudu/127.18.150.188@KRBTEST.COM
I20260618 22:38:22.521838 9601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c8f0ccf6b584d88af43e96e56374695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } } }
I20260618 22:38:22.522125 9601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:22.524864 9604 catalog_manager.cc:1347] Generated new cluster ID: 898a87083d5148d3b2a149a521a265ab
I20260618 22:38:22.524961 9604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:38:22.534483 9599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c8f0ccf6b584d88af43e96e56374695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } } }
I20260618 22:38:22.534757 9599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:22.543969 9616 mvcc.cc:205] Tried to move back new op lower bound from 7298344150996967424 to 7298344150871412736. Current Snapshot: MvccSnapshot[applied={T|T < 7298344150996967424 or (T in {7298344150996967424})}]
I20260618 22:38:22.544791 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.508744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.150.188:36067 (local address 127.0.0.1:52820)
0618 22:38:22.509269 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0618 22:38:22.509289 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:38:22.509413 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:38:22.510052 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:38:22.510058 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:38:22.510584 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:38:22.510781 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.510794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.512315 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.512320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.513696 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:38:22.513710 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.515014 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.515020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:38:22.515231 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.516239 (+ 1008us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:38:22.516265 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:38:22.522487 (+ 6222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:38:22.539329 (+ 16842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.539336 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.539341 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.540260 (+ 919us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.541044 (+ 784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:38:22.541050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:38:22.541054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:38:22.541421 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:38:22.542593 (+ 1172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:38:22.542599 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:38:22.542876 (+ 277us) client_negotiation.cc:769] Sending connection context
0618 22:38:22.543091 (+ 215us) client_negotiation.cc:240] Negotiation successful
0618 22:38:22.543616 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416}
I20260618 22:38:22.548338 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:22.508215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52820 (local address 127.18.150.188:36067)
0618 22:38:22.508471 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.508478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.509537 (+ 1059us) server_negotiation.cc:407] Connection header received
0618 22:38:22.509618 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.509624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.509700 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.509795 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.510992 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.512127 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.513893 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:22.514660 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:22.514781 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:22.522814 (+ 8033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:38:22.522820 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:38:22.522826 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:38:22.522864 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:38:22.539049 (+ 16185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.540542 (+ 1493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.540548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.540552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.540831 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:38:22.541655 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:38:22.541661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:38:22.541664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:38:22.542399 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:38:22.544421 (+ 2022us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:22.544997 (+ 576us) server_negotiation.cc:299] Negotiation successful
0618 22:38:22.547745 (+ 2748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:38:22.553750 9497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119: Bootstrap starting.
I20260618 22:38:22.559801 9497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119: Neither blocks nor log segments found. Creating new log.
I20260618 22:38:22.564512 9497 log.cc:824] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119: Log is configured to *not* fsync() on all Append() calls
I20260618 22:38:22.583853 9604 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:38:22.585251 9497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119: No bootstrap required, opened a new log
I20260618 22:38:22.585578 9604 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:38:22.595817 9497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } }
I20260618 22:38:22.596223 9497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2776e2171478463ba416e25491e1c119, State: Initialized, Role: FOLLOWER
I20260618 22:38:22.596725 9497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } }
I20260618 22:38:22.597987 9621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } } }
I20260618 22:38:22.598752 9621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:22.599174 9497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:38:22.599552 9471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:38:22.603504 9471 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [term 1 FOLLOWER]: Refusing update from remote peer 6c8f0ccf6b584d88af43e96e56374695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:38:22.604746 9601 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:38:22.605518 9472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6c8f0ccf6b584d88af43e96e56374695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2776e2171478463ba416e25491e1c119"
I20260618 22:38:22.609392 9472 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8f0ccf6b584d88af43e96e56374695 in term 1.
I20260618 22:38:22.622296 9604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695: Generated new TSK 0
I20260618 22:38:22.623543 9604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:38:22.696606 9627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c8f0ccf6b584d88af43e96e56374695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } } }
I20260618 22:38:22.697032 9627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:22.696970 9578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c8f0ccf6b584d88af43e96e56374695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } } }
I20260618 22:38:22.697305 9578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c8f0ccf6b584d88af43e96e56374695 [sys.catalog]: This master's current role is: LEADER
I20260618 22:38:22.697832 9621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c8f0ccf6b584d88af43e96e56374695. Latest consensus state: current_term: 1 leader_uuid: "6c8f0ccf6b584d88af43e96e56374695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8f0ccf6b584d88af43e96e56374695" member_type: VOTER last_known_addr { host: "127.18.150.190" port: 33827 } } peers { permanent_uuid: "2776e2171478463ba416e25491e1c119" member_type: VOTER last_known_addr { host: "127.18.150.189" port: 34143 } } peers { permanent_uuid: "35cb7cba29f741a3a87990ea8d399fff" member_type: VOTER last_known_addr { host: "127.18.150.188" port: 36067 } } }
I20260618 22:38:22.698159 9621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:38:22.711833 9638 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260618 22:38:22.752540 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:22.738746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58956 (local address 127.18.150.189:34143)
0618 22:38:22.738949 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.738955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.750576 (+ 11621us) server_negotiation.cc:407] Connection header received
0618 22:38:22.750914 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.750922 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.751015 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.751090 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.752288 (+ 1198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58956: BlockingRecv error: recv got EOF from 127.0.0.1:58956 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260618 22:38:22.753037 9574 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:22.744946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.18.150.188:36067)
0618 22:38:22.745156 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.745162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.751239 (+ 6077us) server_negotiation.cc:407] Connection header received
0618 22:38:22.751413 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.751419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.751499 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.751568 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.752791 (+ 1223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52822: BlockingRecv error: recv got EOF from 127.0.0.1:52822 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260618 22:38:22.753301 9426 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:22.738605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53430 (local address 127.18.150.190:33827)
0618 22:38:22.738782 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.738787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.750684 (+ 11897us) server_negotiation.cc:407] Connection header received
0618 22:38:22.751569 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.751576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.751678 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.751749 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.753068 (+ 1319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53430: BlockingRecv error: recv got EOF from 127.0.0.1:53430 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260618 22:38:22.759744 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:22.757648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58972 (local address 127.18.150.189:34143)
0618 22:38:22.757814 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.757819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.758311 (+ 492us) server_negotiation.cc:407] Connection header received
0618 22:38:22.758507 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.758512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.758641 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.758749 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.759494 (+ 745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58972: BlockingRecv error: recv got EOF from 127.0.0.1:58972 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260618 22:38:22.766726 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:22.764596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58976 (local address 127.18.150.189:34143)
0618 22:38:22.764755 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.764760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.765248 (+ 488us) server_negotiation.cc:407] Connection header received
0618 22:38:22.765426 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.765433 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.765534 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.765642 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.766498 (+ 856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58976: BlockingRecv error: recv got EOF from 127.0.0.1:58976 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260618 22:38:22.781996 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:22.779791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58992 (local address 127.18.150.189:34143)
0618 22:38:22.779971 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.779977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.780463 (+ 486us) server_negotiation.cc:407] Connection header received
0618 22:38:22.780647 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.780653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.780765 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.780887 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.781678 (+ 791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58992: BlockingRecv error: recv got EOF from 127.0.0.1:58992 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260618 22:38:22.827755 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:22.823403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59000 (local address 127.18.150.189:34143)
0618 22:38:22.823645 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.823652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.826447 (+ 2795us) server_negotiation.cc:407] Connection header received
0618 22:38:22.826641 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.826646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.826720 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.826803 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.827538 (+ 735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59000: BlockingRecv error: recv got EOF from 127.0.0.1:59000 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
W20260618 22:38:22.840886 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:22.838858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59014 (local address 127.18.150.189:34143)
0618 22:38:22.839049 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.839056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.839556 (+ 500us) server_negotiation.cc:407] Connection header received
0618 22:38:22.839751 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.839755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.839827 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.839910 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.840665 (+ 755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59014: BlockingRecv error: recv got EOF from 127.0.0.1:59014 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260618 22:38:22.862260 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:22.860166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59022 (local address 127.18.150.189:34143)
0618 22:38:22.860375 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.860383 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.860883 (+ 500us) server_negotiation.cc:407] Connection header received
0618 22:38:22.861078 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.861083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.861170 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.861257 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.862030 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59022: BlockingRecv error: recv got EOF from 127.0.0.1:59022 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260618 22:38:22.900233 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:22.897770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59034 (local address 127.18.150.189:34143)
0618 22:38:22.898035 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.898043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.898712 (+ 669us) server_negotiation.cc:407] Connection header received
0618 22:38:22.898946 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.898974 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.899072 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.899161 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.899960 (+ 799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59034: BlockingRecv error: recv got EOF from 127.0.0.1:59034 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260618 22:38:22.969337 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:22.967546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59038 (local address 127.18.150.189:34143)
0618 22:38:22.967746 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:22.967753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:22.968088 (+ 335us) server_negotiation.cc:407] Connection header received
0618 22:38:22.968262 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:22.968269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:22.968369 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:22.968487 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:22.969138 (+ 651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59038: BlockingRecv error: recv got EOF from 127.0.0.1:59038 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260618 22:38:23.102471 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:23.100624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.18.150.189:34143)
0618 22:38:23.100806 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:23.100813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:23.101179 (+ 366us) server_negotiation.cc:407] Connection header received
0618 22:38:23.101369 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:23.101373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:23.101459 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:23.101545 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:23.102275 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59052: BlockingRecv error: recv got EOF from 127.0.0.1:59052 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260618 22:38:23.363363 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:38:23.361778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.18.150.189:34143)
0618 22:38:23.362011 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:23.362018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:23.362206 (+ 188us) server_negotiation.cc:407] Connection header received
0618 22:38:23.362374 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:23.362379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:23.362455 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:23.362539 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:38:23.363149 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59054: BlockingRecv error: recv got EOF from 127.0.0.1:59054 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260618 22:38:23.494180 9613 catalog_manager.cc:1259] Loaded cluster ID: 898a87083d5148d3b2a149a521a265ab
I20260618 22:38:23.494279 9613 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff: loading cluster ID for follower catalog manager: success
I20260618 22:38:23.496752 9613 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff: acquiring CA information for follower catalog manager: success
I20260618 22:38:23.498560 9613 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 35cb7cba29f741a3a87990ea8d399fff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:38:23.716037 9638 catalog_manager.cc:1259] Loaded cluster ID: 898a87083d5148d3b2a149a521a265ab
I20260618 22:38:23.716158 9638 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119: loading cluster ID for follower catalog manager: success
I20260618 22:38:23.718847 9638 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119: acquiring CA information for follower catalog manager: success
I20260618 22:38:23.720598 9638 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2776e2171478463ba416e25491e1c119: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:38:23.890348 9703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:23.883632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.18.150.189:34143)
0618 22:38:23.884445 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:23.884450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:23.884468 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:38:23.884532 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:23.884536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:23.884673 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:23.884779 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:23.885302 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:23.886129 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:23.888031 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:23.888486 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:23.888573 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:23.889139 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:23.889711 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:23.889810 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:23.889974 (+ 164us) server_negotiation.cc:299] Negotiation successful
0618 22:38:23.890049 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":569,"threads_started":1}
I20260618 22:38:23.902680 9709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:23.896101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.18.150.188:36067)
0618 22:38:23.896581 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:23.896587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:23.896873 (+ 286us) server_negotiation.cc:407] Connection header received
0618 22:38:23.897071 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:23.897075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:23.897222 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:23.897352 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:23.897790 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:23.898580 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:23.900463 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:23.900890 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:23.900979 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:23.901544 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:23.902080 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:23.902182 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:23.902333 (+ 151us) server_negotiation.cc:299] Negotiation successful
0618 22:38:23.902400 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":220,"threads_started":1}
I20260618 22:38:23.927024 9709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:23.908686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52840 (local address 127.18.150.188:36067)
0618 22:38:23.914581 (+ 5895us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:23.914589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:23.919824 (+ 5235us) server_negotiation.cc:407] Connection header received
0618 22:38:23.920134 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:23.920141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:23.920253 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:23.920319 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:23.921069 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:23.922122 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:23.924377 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:23.925051 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:23.925148 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:23.926241 (+ 1093us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:23.926376 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:23.926438 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:23.926648 (+ 210us) server_negotiation.cc:299] Negotiation successful
0618 22:38:23.926732 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5726}
I20260618 22:38:23.927016 9703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:23.914635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59078 (local address 127.18.150.189:34143)
0618 22:38:23.914824 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:23.914831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:23.920077 (+ 5246us) server_negotiation.cc:407] Connection header received
0618 22:38:23.920340 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:23.920347 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:23.920480 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:23.920545 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:23.921261 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:23.922464 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:23.924634 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:23.925317 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:23.925421 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:23.926219 (+ 798us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:23.926354 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:23.926430 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:23.926650 (+ 220us) server_negotiation.cc:299] Negotiation successful
0618 22:38:23.926733 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:38:23.929261 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:38:23.908507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.18.150.190:33827)
0618 22:38:23.921684 (+ 13177us) server_negotiation.cc:206] Beginning negotiation
0618 22:38:23.921690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:38:23.921710 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:38:23.921818 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:38:23.921823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:38:23.922069 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:38:23.922998 (+ 929us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:38:23.923159 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:23.924204 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:23.926606 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:38:23.927178 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:38:23.927782 (+ 604us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:38:23.927961 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:38:23.928385 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:38:23.928860 (+ 475us) server_negotiation.cc:1035] Waiting for connection context
0618 22:38:23.928910 (+ 50us) server_negotiation.cc:299] Negotiation successful
0618 22:38:23.928993 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13047,"thread_start_us":13700,"threads_started":1}
I20260618 22:38:23.933307 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 9356
I20260618 22:38:23.934510 9418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:38:24.043668 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 9356
W20260618 22:38:24.068791 9522 connection.cc:535] server connection from 127.0.0.1:52774 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 22:38:24.069316 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 9430
I20260618 22:38:24.070346 9492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:38:24.174309 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 9430
I20260618 22:38:24.189898 19034 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task8lyoIK/build/asan/bin/kudu pid 9504
I20260618 22:38:24.191002 9566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260618 22:38:24.296718 19034 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8lyoIK/build/asan/bin/kudu with pid 9504
2026-06-18T22:38:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4665 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9035 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (343369 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST